Re: [google-appengine] Re: How to block Google App Engine?

[Barry Hunter]

On 29 March 2011 06:26, Álvaro Degives-Más  wrote:

> It seems this was a difficult question.


Why do you say that?


> For the time being, until I learn a way to specifically block non-Google
> operated bots coming from the Google IP address space, I've blocked any and
> all instances of Googlebot.


One way is via the useragent. All requests coming from AppEngine, clearly
identify themselves in the User-Agent header. There is no way for the
developer to send a custom (or fake) header.



>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Re: How to block Google App Engine?

[Brandon Wirtz]

If you want to block Google AppEngine just block the Appengine User agent in
your HTACESS or equivalent.  Users can’t change the User Agent the way they
do Curl, so if you block the useragent you block all of appengine.

 

For those of you who were wondering why you would do this… Appengine makes a
great proxy and since it comes from 100’s of addresses if you limit an api
call to an IP, an Appengine user can by-pass that limit.  Also because of
the way IP’s are round robin’ed it appears that sometimes AppEngine, and
Google Bot Share Ips so you wouldn’t want to block all of Google, because
your site wouldn’t get indexed.

 

 

From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Álvaro Degives-Más
Sent: Thursday, March 31, 2011 9:46 PM
To: google-appengine@googlegroups.com
Subject: [google-appengine] Re: How to block Google App Engine?

 

Peril travels the other way around: Google App Engine can use index data
from Google. Or other SEs for that matter.

-- 
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.