This my site https://timeforums.appspot.com/forum.php on GAE How to stop the AhrefsBot bot crawl ? I set in *robots.txt* ,
user-agent: AhrefsBot > disallow: / already ,and block ip in *dos.yaml * blacklist: > - subnet: > 188.165.15.238 but , not work .still crawl , How can i do ? Anyone have any other suggestions or not. Thank you. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a9202e1d-bc01-4b53-805e-0ffae7b9ca95%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
