I think in webmaster tools you can change crawl-rate preferences. Otherwise - just block it with robots.txt.
On Sep 3, 12:35 pm, "Brandon Wirtz" <drak...@digerat.com> wrote: > The biggest Problem I have with the Scheduler/GAE isn't GAE it is Google > Bot. > > Under the new model you are on the hook for 15 minutes of time for an > instance that spins up. Google Bot can't be throttled on GAE. If you go in > to WebMasters Tools you get a "Your site has been assigned a special crawl > rate". This was my favorite feature when I was paying for CPU cycles, but > now when Google bot shows up and makes upwards of 100k requests in 5 minutes > every 6 hours, Google Bot will be accounting for about 85% of the cost of > hosting on several of the sites that have less than $50 a month in hosting > costs under the current plan, that will be paying about $400 a month to > serve requests to Google come November Pricing. > > -Brandon -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
