Re: GoDaddy.com shuts down entire data center?
I'm astonished GoDaddy pulled anyone for spamming. Isn't spamming the whole point of GoDaddy, what with its content-free WHOIS records, integrated no-name domain registry and hosting division? In fact, I would go so far as to say taking out entire GoDaddy would probably be a small increase in the amount of useful information on the Net..
Re: GoDaddy.com shuts down entire data center?
Doesn't this fall under bad things happen. Hopefully it is very clear to all on NANOG that DNS changes can have unforeseeable consequences, because of the nature of the delegation in the DNS. As such pulling DNS records (or zones) you don't fully understand the usage of, as a response to a security/spam problem, is generally a bad idea. That said ultimately a decision has to be taken, relative benefits versus risks. I'm very grateful someone arranged that all records used by the MINIT trojan now point to an RFC1918 private address space*, having found infected boxes failing to download their payload as a result. However pulling DNS records probably doesn't belong in the hurly burly of front line support. Simon *Anyone going to check how many DNS servers are still caching asfasf.ath.cx, to tell how many boxes nearly downloaded the payload? In the style of the Sony DRM fiasco measurement.
Re: Odd policy question.
we have appealed to multiple registrars such as godaddy, enom, and the like to remove these bogus NS records from our IP space which keep our new customers from using these IP addresses for hosting but they claim that we have no grounds even though we are the legitimate 'keepers' of said IP space. This is a relatively straightforward issue. The registrars operate according to ICANN policies. Your legitimacy as a keeper of the IP address space descends from ICANN through IANA. Either the registrar is in violation of ICANN policy by not cleaning these NS records, or, the registrar is acting in accordance with ICANN policy. You need to find out which is true and then pester ICANN to either police the registars or fix their broken policy. I suspect that this is something that is not explicit in the ICANN registrar agreements but is implied by some general clause about the wellbeing of the Internet. In that case ICANN would have to issue an interpretation of the situation, pointing out to registrars that cleaning stale NS records is, in fact, part of their ICANN agreement. http://www.icann.org is the place to go. --Michael Dillon
BGP route flap damping
Hi folks, Last week we received a DoS attack which got down my BGP connections to my upstream providers (for three or four times I believe). I also belive that event caused some routers to suppress my BGP announcement. I would appreciate suggestions on how to proceed? with this situation. Thanks in advance. Regards, Gustavo.
Re: BGP route flap damping
On Jan 16, 2006, at 7:28 AM, Gustavo Rodrigues Ramos wrote: Last week we received a DoS attack which got down my BGP connections to my upstream providers (for three or four times I believe). I also belive that event caused some routers to suppress my BGP announcement. I would appreciate suggestions on how to proceed? with this situation. Remind everyone that flap dampening is no longer a good idea, and is in fact considered harmful. (Queue discussion at last RIPE.) The problem is probably not flapping 3 times, but the amplification some people saw. (One of the reasons it was decided not to promote flap dampening at RIPE.) Not much you can do about this in general. In your specific case, since we don't know why your sessions died, we don't know what to suggest to stop it. Perhaps change the timers with your upstream? -- TTFN, patrick
Re: BGP route flap damping
Patrick W. Gilmore wrote: Not much you can do about this in general. In your specific case, since we don't know why your sessions died, we don't know what to suggest to stop it. Perhaps change the timers with your upstream? My BGP connections (and annoucements) with/to my ISPs are all fine. The problem takes place five or six AS far from me... Where I can't do much. I still can't reach some prefixes announced by large ISPs. At the first time, I thought an e-mail to the NOC of the network I can't reach can solve the problem, but it was a waste of time... Thanks again, Gustavo.
Re: BGP route flap damping
The problem takes place five or six AS far from me... Where I can't do much. I still can't reach some prefixes announced by large ISPs. for the movie, see the apnic presentation http://rip.psg.com/~randy/020910.zmao-flap.pdf for the book, see Z. Mao, R. Govindan, G. Varghese, R. Katz Route Flap Damping Exacerbates Internet Routing Convergence 2002 randy
Re: GoDaddy.com shuts down entire data center?
On Mon, 16 Jan 2006, Martin Hannigan wrote: Here's the story on the big outage. http://marc.perkel.com/index.html Here's another recorded conversation. (Can you do this in NJ?) http://marc.perkel.com/audio/godaddy2.mp3 The GoDaddy folks are well trained. Kudos. While I do believe that GoDaddy appears to have some sloppy policies and procedures, if you listen to both conversations, you will find that GoDaddy followed a procedure to deal with the issue, and the caller patently refused to follow it. In my opinion, the caller is just grandstanding, most likely for dramatic effect. I counted over 15 different times when the staff at GoDaddy explained that he needed to follow a specific procedure outlined in an E-mail, and they offered to re-send it as many times as he needed and to whatever E-mail address he wanted. During the conversation, the caller claims that the owner of the Datacenter is too busy trying to move domains to respond to the E-mail that would allow him to resolve the entire issue. If this is the case, then this is really poor priority management, and if what GoDaddy indicates in the call is true (Several warnings and notifications of pending suspension) then I have to wonder what nectartech management was thinking? Furthermore, the caller identifies himself in his blog as a professional asshole, and based on the recorded calls, I have to agree that he has earned his title. -- Vice President of N2Net, a New Age Consulting Service, Inc. Company http://www.n2net.net Where everything clicks into place! KP-216-121-ST
Re: GoDaddy.com shuts down entire data center?
On Mon, 16 Jan 2006, Martin Hannigan wrote: Here's the story on the big outage. http://marc.perkel.com/index.html Here's another recorded conversation. (Can you do this in NJ?) http://marc.perkel.com/audio/godaddy2.mp3 The GoDaddy folks are well trained. Kudos. [ snip ] Furthermore, the caller identifies himself in his blog as a professional asshole, and based on the recorded calls, I have to agree that he has earned his title. As you dig deeper into his site you find out that he does this often for the recorded calls. He's got quite a few to ATT and MCI stored. There's enough there that GoDaddy ought to inquire as to the legality of him taping their call without consent. I don't think the fact that GoDaddy stated they may record is protection for both, but IANAL. This has been debunked well enough to be non operational so we better stop talking about it before we all start getting kook calls and end up as recordings on a website. ;-) -M
Re: GoDaddy.com shuts down entire data center?
Greg Boehnlein wrote: On Mon, 16 Jan 2006, Martin Hannigan wrote: Here's the story on the big outage. http://marc.perkel.com/index.html Here's another recorded conversation. (Can you do this in NJ?) http://marc.perkel.com/audio/godaddy2.mp3 The GoDaddy folks are well trained. Kudos. While I do believe that GoDaddy appears to have some sloppy policies and procedures, if you listen to both conversations, you will find that GoDaddy followed a procedure to deal with the issue, and the caller patently refused to follow it. If I have read it correctly then nectartech has followed the procedures by email after cleaning the phishing computer. But GoDaddy did not ack nectartechs emails. GoDaddy claimed again and again the system was spamming/phishing when in reality the system was switched off. What else could they do? -- Peter and Karin Dambier The Public-Root Consortium Graeffstrasse 14 D-64646 Heppenheim +49(6252)671-788 (Telekom) +49(179)108-3978 (O2 Genion) +49(6252)750-308 (VoIP: sipgate.de) mail: [EMAIL PROTECTED] mail: [EMAIL PROTECTED] http://iason.site.voila.fr/ https://sourceforge.net/projects/iason/
Re: GoDaddy.com shuts down entire data center?
uOn Mon, Jan 16, 2006 at 10:20:23AM -0500, Martin Hannigan wrote: As you dig deeper into his site you find out that he does this often for the recorded calls. He's got quite a few to ATT and MCI stored. There's enough there that GoDaddy ought to inquire as to the legality of him taping their call without consent. I don't think the fact that GoDaddy stated they may record is protection for both, but IANAL. Federal law prohibits private recording of phone calls in the absence of consent from at least one party to the call. Since the caller in this case presumably consented to the recording he was doing, no federal law was broken. Whether or not GoDaddy's we may record statement constitutes consent is irrelevant because their consent is not required. Most state laws are similar to the federal law. Some states, though, require the consent of all the parties to the call. It's not clear what law applies on interstate calls between states with dissimilar laws. In particular, if the caller is in a one-party state and GoDaddy is in an all parties state, then he is potentially violating the law in the all-parties state. Any attempt to prosecute such violation would likely be challanged on the grounds that it was an interstate call so only federal law applies (that is, that the existance of the federal law automatically preempts state law on any interstate call), or on the grounds that there isn't sufficient relationship to GoDaddy's state to allow that state to prosecute the caller. (Put another way, the argument would be that State X is not entitled to regulate what individuals in State Y do with their own phones in State Y, even when they are calling people in state X.) And, of course, if an all-party law were held to apply to this case, then he could argue that he consented and GoDaddy's we might record this call constituted consent for him to record it. In short, if he and GoDaddy are both in the same state, and it's an all-parties state, he probably broke the law (unless he successfulyl argues that GoDaddy effectively consented.) If he and GoDaddy are both in one-party states, he's fine. Anything else, and it's unclear. If his state is one-party, he's probably safe. If his state is all-parties, then it's harder to say, although federal preemption is certainly a reasonable argument to make. http://www.rcfp.org/taping/ seems to have good information. -- Brett
Re: DOS attack against DNS?
[EMAIL PROTECTED] (Mark Andrews) writes: For repeat offenders create a list of networks that won't implement BCP 38 and collectively de-peer with them telling them why you are de-peering and what is required to re-establish connectivity. It is in everyones interests to do the right thing here. people inside one of the largest networks have told me that they have customers who require the ability to bypass BCP38 restrictions, and that they will therefore never be fully BCP38 compliant. i've asked for BCP38 to become the default on all their other present and future customers but then there was whining about bankruptcy, old outdated equipment, and so on. sadly, there's no way to de-peer this network, or any other multinational, and so there will be no peer pressure on them to implement BCP38. so, it's either not in everyone's interests to do the right thing, or there is still a huge variance in what's considered the right thing. either way, we're (the internet is) SCREWED until we (that's we all) fix this. (if you're not seeing spoofed-source attacks, bully for you! i didn't see one today, either, but leaving this tool in the bad-guy toolbox makes us all unsafe, no matter how much or how little they may be using it this day/year.) -- Paul Vixie
Re: DOS attack against DNS?
On Mon, 16 Jan 2006, Paul Vixie wrote: [EMAIL PROTECTED] (Mark Andrews) writes: For repeat offenders create a list of networks that won't implement BCP 38 and collectively de-peer with them telling them why you are de-peering and what is required to re-establish connectivity. It is in everyones interests to do the right thing here. people inside one of the largest networks have told me that they have customers who require the ability to bypass BCP38 restrictions, and that they will therefore never be fully BCP38 compliant. i've asked for BCP38 to become the default on all their other present and future customers but then there was whining about bankruptcy, old outdated equipment, and so on. sadly, there's no way to de-peer this network, or any other multinational, and so there will be no peer pressure on them to implement BCP38. Consider people in the rest of the world who may purchase simplex satellite links. By definition they inject traffic in places they aren't announcing their route from. so, it's either not in everyone's interests to do the right thing, or there is still a huge variance in what's considered the right thing. either way, we're (the internet is) SCREWED until we (that's we all) fix this. (if you're not seeing spoofed-source attacks, bully for you! i didn't see one today, either, but leaving this tool in the bad-guy toolbox makes us all unsafe, no matter how much or how little they may be using it this day/year.) -- -- Joel Jaeggli Unix Consulting [EMAIL PROTECTED] GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
Re: GoDaddy.com shuts down entire data center?
On Sun, Jan 15, 2006 at 03:32:02PM -0800, Matt Ghali wrote: On Sun, 15 Jan 2006, Elijah Savage wrote: Any validatity to this and if so I am suprised that our team has got no calls on not be able to get to certain websites. http://webhostingtalk.com/showthread.php?t=477562 I for one applaud godaddy's response. If more piddling Hosting Providers with Datacenters got turned off when they started spewing abusive traffic, the net would be a much nicer place. Whoever the heck nectartech is, I guess they might act a little more responsibly in the future. Or, more probably, they'll just change to another DNS registrar who doesn't care as much about abuse. FYI, Nectartech is a small hosting shop out of 55 S Market in San Jose. I wouldn't describe them as a datacenter, since I don't think they own or operate any facilities. Perhaps if they ever managed to find the command to make two routers talk to each other and be redundant (a real quote from what has been loosely described as their network admin, I'm not kidding, you can't make stuff like this up :P), their next step might be to find the command to make dns servers talk to each other and be redundant. Reality check time, what we have here is a small hosting shop with a long history of shady customers. I doubt GoDaddy nukes nameservers on a whim, my money is that there was a lot of abuse which went on for a long time without getting any response. Its amazing how quickly some people who don't respond or address abuse issues at all when you're asking nicely will appear and take care of things once you turn them off. The rest is just some random blowhard web hosting customer who gets off on being an ass and blaming everyone but himself and his choice in hosting companies. Hardly an uncommon sight. :) -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: DOS attack against DNS?
[EMAIL PROTECTED] (Joel Jaeggli) writes: people inside one of the largest networks have told me that they have customers who require the ability to bypass BCP38 restrictions, and that they will therefore never be fully BCP38 compliant. ... Consider people in the rest of the world who may purchase simplex satellite links. By definition they inject traffic in places they aren't announcing their route from. yup, those are exactly the customers i was told about. (see above.) however, there's still a way to filter-list the various interfaces -- it's just harder than letting the routing table imply your filter-list for you. also however, if these were the only customers who weren't made to follow BCP38, there would not be a global BCP38-related problem right now. or, as i said before: i've asked for BCP38 to become the default on all their other present and future customers ... -- Paul Vixie
Re: BGP route flap damping
On Jan 16, 2006, at 8:48 AM, Gustavo Rodrigues Ramos wrote: Patrick W. Gilmore wrote: Not much you can do about this in general. In your specific case, since we don't know why your sessions died, we don't know what to suggest to stop it. Perhaps change the timers with your upstream? My BGP connections (and annoucements) with/to my ISPs are all fine. The problem takes place five or six AS far from me... Where I can't do much. I still can't reach some prefixes announced by large ISPs. At the first time, I thought an e-mail to the NOC of the network I can't reach can solve the problem, but it was a waste of time... I'm a little confused. Are you saying you dampened the prefixes of some other network? If so, it sounds like this is 100% in your control. If the BGP sessions between you and your upstreams / peers never flapped, no one should have dampened you. (I can see it possibly happening if someone else in the path between you and $OtherNetwork is attacked and therefore flaps your routes, but that would affect a lot of networks, not just you.) -- TTFN, patrick
Re: GoDaddy.com shuts down entire data center?
Richard, On the other hand , I'm not comfortable with the idea that an organization that provides network infrastructure services under the aegis of the US Government could unilaterally revoke those services for something that is not illegal. By all means, the Justice Dept. and police should move against anyone performing illegal acts such as phishing, I just don't think that it is ICANN or ARIN and GoDaddy's job to police good net citizenship. Joe On 1/16/06 10:07 AM, Richard A Steenbergen [EMAIL PROTECTED] wrote: On Sun, Jan 15, 2006 at 03:32:02PM -0800, Matt Ghali wrote: On Sun, 15 Jan 2006, Elijah Savage wrote: Any validatity to this and if so I am suprised that our team has got no calls on not be able to get to certain websites. http://webhostingtalk.com/showthread.php?t=477562 I for one applaud godaddy's response. If more piddling Hosting Providers with Datacenters got turned off when they started spewing abusive traffic, the net would be a much nicer place. Whoever the heck nectartech is, I guess they might act a little more responsibly in the future. Or, more probably, they'll just change to another DNS registrar who doesn't care as much about abuse. FYI, Nectartech is a small hosting shop out of 55 S Market in San Jose. I wouldn't describe them as a datacenter, since I don't think they own or operate any facilities. Perhaps if they ever managed to find the command to make two routers talk to each other and be redundant (a real quote from what has been loosely described as their network admin, I'm not kidding, you can't make stuff like this up :P), their next step might be to find the command to make dns servers talk to each other and be redundant. Reality check time, what we have here is a small hosting shop with a long history of shady customers. I doubt GoDaddy nukes nameservers on a whim, my money is that there was a lot of abuse which went on for a long time without getting any response. Its amazing how quickly some people who don't respond or address abuse issues at all when you're asking nicely will appear and take care of things once you turn them off. The rest is just some random blowhard web hosting customer who gets off on being an ass and blaming everyone but himself and his choice in hosting companies. Hardly an uncommon sight. :) -- Joe McGuckin ViaNet Communications 994 San Antonio Road Palo Alto, CA 94303 Phone: 650-213-1302 Cell: 650-207-0372 Fax: 650-969-2124
Re: GoDaddy.com shuts down entire data center?
On Mon, 16 Jan 2006, Richard A Steenbergen wrote: FYI, Nectartech is a small hosting shop out of 55 S Market in San Jose. I wouldn't describe them as a datacenter, since I don't think they own or operate any facilities. Heh, I used to work at a small hosting shop out of 55 S. Market- it was (then) called BBN Planet. I guess these schmoes rent a cage from Genuity (or whatever they are called now). Perhaps if they ever managed to find the command to make two routers talk to each other and be redundant (a real quote from what has been loosely described as their network admin, I'm not kidding, you can't make stuff like this up :P), their next step might be to find the command to make dns servers talk to each other and be redundant. Seriously. You need to be spewing a lot of cak onto the net for your _domain registrar_ to take notice. The rest is just some random blowhard web hosting customer who gets off on being an ass and blaming everyone but himself and his choice in hosting companies. Hardly an uncommon sight. :) The priceless part is that we probably never would have noticed, had he not had the hubris to record the conversations, and then publish the URL to them. I love it when the lusers are nice enough to clearly identify themselves. matto [EMAIL PROTECTED]darwin The only thing necessary for the triumph of evil is for good men to do nothing. - Edmund Burke
sc minutes of 2006.01.12
marty hannigan asked that we make on-list noise when sc minutes are posted. so this is the noise. http://www.nanog.org/sc.minutes06.html randy
Re: GoDaddy.com shuts down entire data center?
On Mon, 16 Jan 2006, Joe McGuckin wrote: Richard, On the other hand , I'm not comfortable with the idea that an organization that provides network infrastructure services under the aegis of the US Government could unilaterally revoke those services for something that is not illegal. It does not have to be illegal. All that is necessary is that customer who purchased the service beware and agree to the policies prior to making the purchase (of course, almost nobody fully reads that long agreement you get presented on the website, but that's another story...) Not being somebody who've ever used godaddy's services, I'm just speculating based on various reports, but I think their registration service agreement is more extensive then domain registration agreement from most other registrars and prohibits use of the domain in connection with spamming as well as in connection with illegal activities. If policies are violated then domain maybe suspended until problem is resolved. I suspect they don't suspend right away and have system of requiring domain owner be available for notification and conversation in case such use (prohibited by their service agreement) is reported. If they do not hear anything about it and reports continue then they take action as allowed by domain registration agreement. What we probably saw is such action after nectartech failed to respond to several notifications and probably kept server running without fully cleaning it up and possibly more then one of their servers was hacked too. This is similar enough situation to what may happen when you run servers on the connection purchased from your ISP and that ISP actually takes abuse reports seriously and has working abuse department that follows up on what is sent them. That this was spinned around as datacenter shutdown on WHT and even got here is a result of both how nectartech wanted itself seen and who they had for dealing with such vendor actions. On Mon, 16 Jan 2006, Richard A Steenbergen wrote: The rest is just some random blowhard web hosting customer I disagree with this particular part. I think its quite clear that this was not random blowhard hosting customer but somebody close to nectartech owner who owner knew could get through walls put by some companies and if not annoy the hell out of them afterward and spin it around in [in]appropriate way. -- William Leibzon Elan Networks [EMAIL PROTECTED]
Re: DOS attack against DNS?
At 12:52 PM 1/16/2006, Joel Jaeggli wrote: On Mon, 16 Jan 2006, Paul Vixie wrote: [EMAIL PROTECTED] (Mark Andrews) writes: For repeat offenders create a list of networks that won't implement BCP 38 and collectively de-peer with them telling them why you are de-peering and what is required to re-establish connectivity. It is in everyones interests to do the right thing here. people inside one of the largest networks have told me that they have customers who require the ability to bypass BCP38 restrictions, and that they will therefore never be fully BCP38 compliant. i've asked for BCP38 to become the default on all their other present and future customers but then there was whining about bankruptcy, old outdated equipment, and so on. sadly, there's no way to de-peer this network, or any other multinational, and so there will be no peer pressure on them to implement BCP38. Consider people in the rest of the world who may purchase simplex satellite links. By definition they inject traffic in places they aren't announcing their route from. Sounds like the landing sites would not be able to use Unicast RPF. However, they could still use BCP38. Nothing says the filters have to be magically generated from routing data (not that uRPF really does that either, since it works off the FIB on most routers). Mobile IP had the same set of issues when we were first working on the ingress filtering drafts. In their case, a bit of tunneling solved the issue. While tunneling could easily solve the satellite case too, there may be resistance to that.
Re: GoDaddy.com shuts down entire data center?
william(at)elan.net wrote: On Mon, 16 Jan 2006, Richard A Steenbergen wrote: The rest is just some random blowhard web hosting customer I disagree with this particular part. I think its quite clear that this was not random blowhard hosting customer but somebody close to nectartech owner who owner knew could get through walls put by some companies and if not annoy the hell out of them afterward and spin it around in [in]appropriate way. Precisely. It wasn't just some random blowhard web hosting customer. It was a carefully selected web hosting customer specifically chosen for his expertise at being a blowhard. -- Jay Hennigan - CCIE #7880 - Network Administration - [EMAIL PROTECTED] NetLojix Communications, Inc. - http://www.netlojix.com/ WestNet: Connecting you to the planet. 805 884-6323
Re: GoDaddy.com shuts down entire data center?
william(at)elan.net wrote: On Mon, 16 Jan 2006, Richard A Steenbergen wrote: The rest is just some random blowhard web hosting customer I disagree with this particular part. I think its quite clear that this was not random blowhard hosting customer but somebody close to nectartech owner who owner knew could get through walls put by some companies and if not annoy the hell out of them afterward and spin it around in [in]appropriate way. Precisely. It wasn't just some random blowhard web hosting customer. It was a carefully selected web hosting customer specifically chosen for his expertise at being a blowhard. He sounds like a blowhard to me, and he delayed them getting back online as quick as he could. GoDaddy gave him the same sphiel I've heard 100 times i.e. here's our procedures please do x, y, and z. If you look at the guys web page, he takes pride in being a blow hard so don't fret, he'd disagree with you too. No doubt he's reading NANOG and probably yelling at the mailing admins about how he has to sign up for two lists vs. one and how stupid we all are. -M
Marc, care to respond?
I agree that in fact it seems that Go daddy followed their procedures to the letter as they are supposed to do. The gentlemen in The Presidents office was very concise in his assessment of the issue, AND his repeated attempts to take the issue home to closure. I used to work for a guy that ran a datacenter in a similar way, and had to make calls on his behalf to carriers to plead my case, after emails to the principle went unanswered. I can feel the pain that Marc was under in his pleas, and hope that I have never been so whiney, or belligerent with my carriers. After reading his home page I see there is a bit of leaning towards aggrandizement, so with a balance to all: Marc would you care to respond now that our feet are out of the fire? The repeated use of the phrase that an entire datacenter is turned off is rather distressing, given that all that was turned off where the links to the primary name servers? We operate in a datacenter, and provide name services to our clients. Often, we have to shut down, remove from network, a non-managed server, that is spewing crap. Usually we are the ones to discover this, but if our carriers report this, we have to respond quickly, that is the business we are all in. Marc, are you the technical contact for the domain, or a customer? If you are the technical contact, I hope that you will be more careful, or make your email the primary technical contact for the domain. If you are just a friend, and I am guessing customer, for this domain, RUN I use Go daddy, and the same procedures that you where trying to circumvent, are the same procedures that they would have gone through to notify the POC. Mark D. Bodley Senior Partner, Cyrix Systems 954-537-9499 [EMAIL PROTECTED]
Re: DOS attack against DNS?
In article [EMAIL PROTECTED] you write: On Mon, 16 Jan 2006, Paul Vixie wrote: [EMAIL PROTECTED] (Mark Andrews) writes: For repeat offenders create a list of networks that won't implement BCP 38 and collectively de-peer with them telling them why you are de-peering and what is required to re-establish connectivity. It is in everyones interests to do the right thing here. people inside one of the largest networks have told me that they have customers who require the ability to bypass BCP38 restrictions, and that they will therefore never be fully BCP38 compliant. i've asked for BCP38 to become the default on all their other present and future customers but then there was whining about bankruptcy, old outdated equipment, and so on. sadly, there's no way to de-peer this network, or any other multinational, and so there will be no peer pressure on them to implement BCP38. Consider people in the rest of the world who may purchase simplex satellite links. By definition they inject traffic in places they aren't announcing their route from. But they don't need to be able to source all of 0/0. They need to be able to source particular addresses which they have. If the end point of the satellite link is dynamic then they need to souce netblocks. The satellite company should be able to supply a complete list so filters can be setup appropriately. BCP 38 isn't all or nothing. You do the best you can. You limit the exposure. In this case if you get spoofed traffic from the satellite company's addresses you still talk to the satellite company to address the problem. If they have static address assignment it should be a easy job to trace the offending traffic back. If they have dynamic assignment then things get harder. It should be possible to prevent any owned box (other than a router) spewing out spoofed traffic to the net as a whole. owned routers are a different kettle of fish. This is not a new problem. Sooner or later goverments will mandate this sort of filtering if the networking community as a whole don't do it and they may not leave room to support satellite down links. Think manditory strict unicast reverse path filtering everywhere. so, it's either not in everyone's interests to do the right thing, or there is still a huge variance in what's considered the right thing. either way, we're (the internet is) SCREWED until we (that's we all) fix this. (if you're not seeing spoofed-source attacks, bully for you! i didn't see one today, either, but leaving this tool in the bad-guy toolbox makes us all unsafe, no matter how much or how little they may be using it this day/year.) -- -- Joel Jaeggli Unix Consulting [EMAIL PROTECTED] GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2
Re: GoDaddy.com shuts down entire data center?
On Sun, 15 Jan 2006, Elijah Savage wrote: Any validatity to this and if so I am suprised that our team has got no calls on not be able to get to certain websites. http://webhostingtalk.com/showthread.php?t=477562 Casting blame may be a fun exercise. Listening to others cast blame gets old fast. The more useful question here is whether there are lessons the rest of us can learn from this incident. The most important lesson is probably that your problems will almost always be more important to you than to somebody else. If you end up with a business killing problem, it doesn't matter if it's somebody else's fault -- you're the one who will be out of business. Likewise, you shouldn't go wandering out into heavy traffic just because the drivers are required by law to stop for you. Choosing your vendors carefully is important. Having a backup plan for what to do if your vendors fail you is a good thing, but it's nice not to have to use the backup plan. Likewise, if something is really important to you, make sure your vendors know that. Nobody wants to suddenly find out in the middle of the night that they're responsible for something critical. Knowing what's important to you in advance can help you figure out what arrangements need to be made. If your hosting operation won't run without power, Internet connectivity, and DNS, making sure your power, connectivity, and DNS are robust matters a lot. If your business can continue to operate for a few days without toner for your laser printer, choosing a less reliable toner supplier is probably ok. If you do need to call your vendors, having a clear explanation of what's going on is often a good thing. An entire datacenter is an awfully vague term. If that were all of, say, Equinix Ashburn, it would be a big enough deal that government regulators would probably be concerned. But a room in the back of somebody's office with a rack of servers in it could also be justifiably called a datacenter (and a rack of servers in the back of somebody's office could also be important to somebody). It's probably better to be able to say, x number of domains are down, representing y amount of revenue for our company and z critical service that the rest of the Internet relys on. This might put us out of business. This still may not get the desired response -- it's not your vendor who is going to be put out of business -- but it at least gives the person on the other end of the phone call some idea of what they're dealing with. Protecting everything you've decided is important may be expensive. It may not be worth the cost. It's best to have made that calculation before the problem starts, when there's still time to spend money on protection if you do decide it's worth it. Not having all your DNS servers in the same domain, or registered through the same registrar, isn't a best practice that has previously occurred to me, but it makes a lot of sense now that I think about it. Looking at the big TLDs, .com and .net have all their servers in the gtld-servers.net domain, but Verisign controls .net and can presumably fix gtld-servers.net if it breaks. UltraDNS has their TLD servers (for .org and others) in several different TLDs. Maybe that is to protect against this sort of thing. And there's a PR lesson here, too. I'd never heard of Nectartech before this, and I'm guessing that's the case for a lot of NANOG readers. Having heard this story, I'd be hesitant to register a domain with GoDaddy, and that was presumably the goal. But I'd be hesitant to rely on a company with a name like GoDaddy anyway, just because of the name. Now that I've heard of Nectartech, I know them as the company that had the outage. That's not exactly a selling point. I've certainly got sympathy for Mr. Perkel. I've learned a lot of the lessons above the hard way, some due to my own miscalculations and some due to working for companies that didn't value my time and stress levels as highly as I would have liked (choosing your employers carefully is important too...). These lessons don't apply just to networking. The loss prevention department of a bank once locked my account for suspicious activity on a Friday afternoon and then left for the weekend. I had two dollars in my wallet, and didn't have much food. Escalating as far as I could through the ranks of people working the bank's customer service lines on Friday evening, I didn't manage to find anybody who didn't think I should just wait until Monday. Multiple accounts at different banks, neither of which is the bank that locked my account, now seem like a very good idea. -Steve
Re: DOS attack against DNS?
Not true,. the ANY query has mutliple uses for consolidating multiple diagnostic queries into a single display, and also for diversion monitoring systems on small domains or groups of same. Not all of us have the resources (or time) of large ISPs behind us. On 15 Jan 2006 17:27:40 +, Paul Vixie [EMAIL PROTECTED] wrote: client xx.xx.xx.xx#6704: query: z.tn.co.za ANY ANY +Eclass ANY has no purpose in the real world, not even for debugging.ifyou see it in a query, you can assume malicious intent.if you hear it in a query, you can safely ignore that query, or at best, map it to class IN.--Paul Vixie
Re: DOS attack against DNS?
Did you notice that it was class ANY and not type ANY that Paul noted? I've never ever heard of it being used anywhere As for ANY query type, what do you think will happen when you query with ANY to a host in a domain that is not in your local dns server cache? And btw if it is in your dns cache, how predictable do you think such results are going to be??? On Tue, 17 Jan 2006, Alon Tirosh wrote: Not true,. the ANY query has mutliple uses for consolidating multiple diagnostic queries into a single display, and also for diversion monitoring systems on small domains or groups of same. Not all of us have the resources (or time) of large ISPs behind us. On 15 Jan 2006 17:27:40 +, Paul Vixie [EMAIL PROTECTED] wrote: client xx.xx.xx.xx#6704: query: z.tn.co.za ANY ANY +E class ANY has no purpose in the real world, not even for debugging. if you see it in a query, you can assume malicious intent. if you hear it in a query, you can safely ignore that query, or at best, map it to class IN. -- Paul Vixie
Re: DOS attack against DNS?
Admitted, i did not notice the type/class difference. I responded as a knee jerk reaction, and that is my mistake.For the second part, the any query type is useful (when targeted at either your NS and/or public NS servers) to quickly alert to issues such as the one being discussed with GoDaddy and Nectartech right now on this list. Pick and/or set up an NS server that is TTL agnostic (flameArmor: this system is to be used for disparate up-to-date checks only, and I know by spec this is far from foolproof but its saved my ass a couple times in the past) and checks disparate roots and its useful for finding or alerting to major name system, registrar ,and provider issues quickly. Im diverging off-topic, im sure. gnight.On 1/17/06, william(at)elan.net [EMAIL PROTECTED] wrote: Did you notice that it was class ANY and not type ANY that Paul noted? I've never ever heard of it being used anywhereAs for ANY query type, what do you think will happen when you query withANY to a host in a domain that is not in your local dns server cache? And btw if it is in your dns cache, how predictable do you think suchresults are going to be???On Tue, 17 Jan 2006, Alon Tirosh wrote: Not true,. the ANY query has mutliple uses for consolidating multiple diagnostic queries into a single display, and also for diversion monitoring systems on small domains or groups of same. Not all of us have the resources (or time) of large ISPs behind us. On 15 Jan 2006 17:27:40 +, Paul Vixie [EMAIL PROTECTED] wrote: client xx.xx.xx.xx#6704: query: z.tn.co.za ANY ANY +E class ANY has no purpose in the real world, not even for debugging.if you see it in a query, you can assume malicious intent.if you hear it in a query, you can safely ignore that query, or at best, map it to class IN. -- Paul Vixie
Re: GoDaddy.com shuts down entire data center?
I want to say, from an outsider's perspective, that I whole heartily applaud GoDaddy on the actions they took and the consistent professionalism exhibited by their tech support representative. Despite obvious (and heavily edited) calls to the same agent, the consumer was informed in a professional manner of his/her avenue for resolution. No doubt remains in my mind that the caller was not caught blind by this situation. Go Daddy has a privacy policy that no doubt prohibits them from releasing details of their side of this case, however to me the recording suggests that the caller knew this was the end result, not a sudden surprise move, and they just wanted to circumvent standard proceedure. The caller's prior thought to record, what appears as a standard call to tech-support, is insightful and should be an obvious sign of his motivation. Let me explain my perspective. I am a long standing customer of data center services, and I fully appreciate network operators' efforts to stem the spread of spam and viruses. I run a few non-profit public mailing lists and the emails from my systems traverse your networks hourly. I work quikly and diligently with service providers to overcome issues where our paths cross. I have never been a Go Daddy customer, but I certainly appreciate their stand on this issue. I will probably never be a Nectartech customer after this episode. -Jim P. - Original Message From: william(at)elan.net [EMAIL PROTECTED] To: Joe McGuckin [EMAIL PROTECTED] Cc: Richard A Steenbergen [EMAIL PROTECTED]; Matt Ghali [EMAIL PROTECTED]; Elijah Savage [EMAIL PROTECTED]; NANOG nanog@merit.edu Sent: Monday, January 16, 2006 3:43:53 PM Subject: Re: GoDaddy.com shuts down entire data center? On Mon, 16 Jan 2006, Joe McGuckin wrote: Richard, On the other hand , I'm not comfortable with the idea that an organization that provides network infrastructure services under the aegis of the US Government could unilaterally revoke those services for something that is not illegal. It does not have to be illegal. All that is necessary is that customer who purchased the service beware and agree to the policies prior to making the purchase (of course, almost nobody fully reads that long agreement you get presented on the website, but that's another story...)
Re: GoDaddy.com shuts down entire data center?
On Jan 17, 2006, at 1:32 AM, Jim Popovitch wrote: I want to say, from an outsider's perspective, that I whole heartily applaud GoDaddy on the actions they took [...] There seems to be a wide split on this topic. I was wondering if people would privately tell me yes or no on a few questions so I can understand the issue better. 1) Do you think it is acceptable to cause any collateral damage to innocent bystanders if it will stop network abuse? 2) If yes, do you still think it is acceptable to take down 100s of innocent bystanders because one customer of a provider is misbehaving? 3) If yes, do you still think it is acceptable if the misbehaving customer is not intentionally misbehaving - i.e. they've been hacked? 3) If yes, do you still think it is acceptable if the collateral damage (taking out 100s of innocent businesses) doesn't actually stop the spam run / DoS attack / etc.? These are important question to me, and I'm surprised at the number of people who seem to feel so very differently than I thought they would feel - than I personally feel. Would people mind sending me private e-mails with yes/no answers? Longer answers are welcome, but yes/no will do. Using the case under discussion as an example, I am wondering why anyone thinks taking down 100s of innocent domains is a good way to stop a single hacked machine from doing whatever it is doing? If you somehow think all that is worth it, take a close look at your cost / benefit analysis. At this rate, every business on the Internet will be out of business before we take out even a single moderately large botnet. I am also wondering why anyone thinks the miscreant will stop just because the legitimate owner's domain no longer resolves? Not only is the machine likely to continue sending spam as if nothing happened, we aren't even catching the guy. I guess you could say well, it put pressure on his hosting provider to clean the infected machine, which is true. I just think that's a bit silly. But maybe I'm the one who's silly. Lastly, I wonder what average people - people who run businesses on hosting providers who really don't understand all this computer stuff - think about such actions. How many 100s of people have we just alienated for life to stop - er, NOT stop - a single zombie? And how many of their friends are going to hear over an over how the Internet is not a real business and no one should put any faith in it? Is this really a good thing? -- TTFN, patrick
Re: GoDaddy.com shuts down entire data center?
On Mon, 16 Jan 2006, Jim Popovitch wrote: [jim, please wrap your text!] I have never been a Go Daddy customer, but I certainly appreciate their stand on this issue. I will probably never be a Nectartech customer after this episode. Hear Hear. After reading the GoDaddy domain registration legal agreement, available at: https://www.godaddy.com/gdshop/legal_agreements/show_doc.asp?se=%2Bci=1839pageid=REG%5FSA especially section 7, Restriction of Services, Right of Refusal, I have to give them a big thumbs up. It is good to see that wielding a Big Stick, and actively working for the Good Guys has not hindered GoDaddy from achieving quite a bit of success in the market. matto [EMAIL PROTECTED]darwin The only thing necessary for the triumph of evil is for good men to do nothing. - Edmund Burke
