I wrote this little gist to try and solve the issue you're experiencing: https://gist.github.com/836228
It needs a lot of testing, and assumes that the time is synced on the servers google uses to serve robots.txt. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.