Set your app to know the URL and not work at the appspot.com address. I did this so that things wouldn't get indexed there, or use a robots.txt that varies by Domain.
-----Original Message----- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of MLTrim Sent: Wednesday, October 12, 2011 1:08 AM To: Google App Engine Subject: [google-appengine] Datastore reads high because Googlebot crawls both foo.appspot.com and www.foo.com Hi I'm trying to optimize with memcache/instance cache/proxy cache a gae web application to stay in the free 0.05 Datastore reads range without success. The culprit? Googlebot. Checking the application logs I see that googlebot is fetching both foo.appspot.com and www.foo.com doubling the traffic of the crawling requests that implies double datastore reads (that I would like to limit). I'm already serving via application a 301 status on every get to foo.appspot.com redirecting the traffic to www.foo.com but googlebot still insist to crawl on appspot. >From Webdeveloper tools is not even possible to alert google that your application is moved to another URL because that options is limited just for the root level domains. Do you have any solution to suggest? Cross-post on Webmaster central: http://www.google.com/support/forum/p/Webmasters/thread?tid=21913d25c29aa2ce &hl=en&fid=21913d25c29aa2ce0004af152a88c076 thanks Michele [systempuntoout] http://www.gaecupboard.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.