I noticed today that "TweetmemeBot", a web crawler, was making lots of requests to my tools and ignoring the robots.txt file. The thing scrapes at a very high rate, sometimes over 30 requests per minute.
Other tool maintainers (especially geohack) may want to investigate whether this is also impacting their tools, and explicitly block the user agent if it is. - Carl _______________________________________________ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
