Set your app to know the URL and not work at the appspot.com address. I did
this so that things wouldn't get indexed there, or use a robots.txt that
varies by Domain.
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of MLTrim
Sent: Wednesday, October 12, 2011 1:08 AM
To: Google App Engine
Subject: [google-appengine] Datastore reads high because Googlebot crawls
both foo.appspot.com and www.foo.com
Hi
I'm trying to optimize with memcache/instance cache/proxy cache a gae web
application to stay in the free 0.05 Datastore reads range without success.
The culprit? Googlebot.
Checking the application logs I see that googlebot is fetching both
foo.appspot.com and www.foo.com doubling the traffic of the crawling
requests that implies double datastore reads (that I would like to limit).
I'm already serving via application a 301 status on every get to
foo.appspot.com redirecting the traffic to www.foo.com but googlebot still
insist to crawl on appspot.
>From Webdeveloper tools is not even possible to alert google that your
application is moved to another URL because that options is limited just for
the root level domains.
Do you have any solution to suggest?
Cross-post on Webmaster central:
http://www.google.com/support/forum/p/Webmasters/thread?tid=21913d25c29aa2ce
&hl=en&fid=21913d25c29aa2ce0004af152a88c076
thanks
Michele
[systempuntoout]
http://www.gaecupboard.com
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
