Faidon Liambotis has uploaded a new change for review.
https://gerrit.wikimedia.org/r/141386
Change subject: mailman: block ArchiveTeam ArchiveBot UA
......................................................................
mailman: block ArchiveTeam ArchiveBot UA
Crawling us for hours, on URLs that are disallowed via our robots.txt.
Change-Id: I7e8f187d2ce0edd43873d819f9d71f8a8bf5d2c7
---
M files/lighttpd/50-mailman.conf
1 file changed, 5 insertions(+), 0 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/86/141386/1
diff --git a/files/lighttpd/50-mailman.conf b/files/lighttpd/50-mailman.conf
index 45d1c3c..fb92d5e 100644
--- a/files/lighttpd/50-mailman.conf
+++ b/files/lighttpd/50-mailman.conf
@@ -76,3 +76,8 @@
$HTTP["url"] =~ "^/mailman/(private|admin)/arbcom-l\.mbox(/|$)" {
url.access-deny = ("")
}
+
+# Does not obey robots.txt; causing operational issues 2014-06-23
+$HTTP["useragent"] =~ "ArchiveTeam ArchiveBot" {
+ url.access-deny = ( "" )
+}
--
To view, visit https://gerrit.wikimedia.org/r/141386
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7e8f187d2ce0edd43873d819f9d71f8a8bf5d2c7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis <fai...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
