I don't have solution other than to block requests by agent.
And I really only provided this information so that someone at GOOG could go track him down and beat him (again). From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Viper Sent: Monday, August 08, 2011 2:28 PM To: google-appengine@googlegroups.com Cc: carolinasviperc...@gmail.com Subject: Re: RE: [google-appengine] Is an APP ENGINE Application Attacking websites? - DOTS user agent / browser Ah, finally some information. Ok, so now we know that this is a "not so good" Google project as this spider definitely doesn't behave itself - what can be done? As it stands right now, 75.125 is completely blocked because this thing is hitting from IP's all across the range. The problem is, the block is affecting other things like Google Web Preview, etc. Also, is there a reason that a webmaster would want Google to crawl pages for the web preview (or not want) as it appears that the bot also doesn't necessarily honor robots.txt (as reported in other posts) What else comes from 75.125.X.X Why would projects such as Google DOTS that have potential to cause these sort off issues be permitted to run from IP addresses where Google App Engine traffic and other Google Spider traffic originate? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/zQ0_X02n0BEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
