Any chance its not a "real" AhrefsBot and someone else isn't just using there User Agent?
Ant On Thursday, 22 January 2015 04:49:18 UTC, ASIA IO wrote: > > This my site https://timeforums.appspot.com/forum.php on GAE > How to stop the AhrefsBot bot crawl ? > I set in *robots.txt* , > > user-agent: AhrefsBot >> disallow: / > > already ,and block ip in *dos.yaml * > blacklist: > >> - subnet: >> 188.165.15.238 > > > but , not work .still crawl , > How can i do ? Anyone have any other suggestions or not. > Thank you. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/83d0e761-d916-445e-9e3f-fc0ff009f50b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
