Re: Retrospective freeze break: Block riddler.io
+1 great work tracking it down. kevin pgpiJiPiCHNqI.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Retrospective freeze break: Block riddler.io
On Fri, Apr 17, 2015 at 01:36:50PM +0200, Patrick Uiterwijk wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, Earlier today, the riddler.io bot pulled Bodhi under by performing a lot of heavy requests. I know from experience that this bot does NOT listen to robots.txt, even though the website says it does (it hit my personal webserver as well at an earlier time), so I have currently blocked it by user agent. Is this something we could report to them? 1 files changed, 3 insertions(+), 1 deletions(-) [puiterwijk@lockbox01 files]$ git show HEAD commit a635e91849f29dd3c146b8067f602871ff959f00 Author: Patrick Uiterwijk puiterw...@redhat.com Date: Fri Apr 17 11:29:48 2015 + Block the Riddler.io bot from accessing bodhi This bot does NOT follow robots.txt, even though it announces that it does, and it hits Bodhi so much that it pulls the bodhi servers under. An email has been sent to the maintainer of the bot, but for the time being, let's block it from using any bodhi resources. Signed-off-by: Patrick Uiterwijk puiterw...@redhat.com diff --git a/roles/bodhi/base/files/bodhi-app.conf b/roles/bodhi/base/files/bodhi-app.conf index 3e10a59..3f44964 100644 - --- a/roles/bodhi/base/files/bodhi-app.conf +++ b/roles/bodhi/base/files/bodhi-app.conf @@ -13,6 +13,8 @@ Alias /updates/tg_widgets/tgmochikit/packed/MochiKit/MochiKit.js /usr/lib/python Directory /usr/share/bodhi WSGIProcessGroup bodhi - - Order deny,allow + SetEnvIf User-Agent Riddler GoAway=1 + Order allow,deny Allow from all + Deny from env=GoAway /Directory +1 for me, thanks! Pierre pgpuQtNyM1Nwl.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Retrospective freeze break: Block riddler.io
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Is this something we could report to them? Yup, we can, and I have. I will post to the list if I get a reply. +1 for me, thanks! Thanks! Pierre Patrick -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBCgAGBQJVMRA1AAoJEIZXmA2atR5QwWEP/Ajpjm28nRFypZaTvte4e+PB 34orDF9lYy/Zw3a+0rom0WvZcRvX0Pj6+DFB9j4wOk2+WsL5vwuNBeFxIc71vqj7 CwsX7WVEBwTthkonpegBmuTsnFeu8nVUvv7apU4oXuIbhXpaY7M0IisdntH37wxi 7d+BiNZIAqMdaHNzCWJJyp3Rr/Fi25MMT0OQftjoiaxctXGmHZMMRKI5Ci2X0JSA pYDx1v14RiuIP3xEotVwhBk0VcM9GYZtKJEHfWJHfB+iPJu+c23TmecBFqDvsPCy sdanYDrfcCZtXe1Oq7+dK01V1J7JA2HMadI4veZT0vvqmnXRMquqZzw+cKFLpD/z CoIkYqmCGmcZ2YrrttwdVT8eX7fGB90p8Y9W+1dcr+pZ3C1m0ZgFtPO2TaGHMbea JhItKJSbcRrDtUgHBw+CLm+r3ynm8X2zRuMiHmdEDTzLn/YKOiReGx9L0eZyEA7i VjH0JMQ5uJSHcjrnGC1TiKZymR+4QNtWR3UOsdj9fPrzT0fhUQ8G8hYNgAeoUqXY OVsOfMvIPUqSUxI71WMynrp8VaC+FYcqJYp44AcfQXAfaO+UyVQNj9FUgKL6s8W7 Ng55eFM7WfWdxHjxF816YeiOmrZuHmkP3Q1LQ8OVTip8DeOkqBBdAmNWo1EsdbLf dGGjnESA6eP8fO2vTO/5 =XaNW -END PGP SIGNATURE- ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Retrospective freeze break: Block riddler.io
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, Earlier today, the riddler.io bot pulled Bodhi under by performing a lot of heavy requests. I know from experience that this bot does NOT listen to robots.txt, even though the website says it does (it hit my personal webserver as well at an earlier time), so I have currently blocked it by user agent. We probably need to fix this better soon, but for the time being I put this in to make Bodhi behave again. Any comments? 1 files changed, 3 insertions(+), 1 deletions(-) [puiterwijk@lockbox01 files]$ git show HEAD commit a635e91849f29dd3c146b8067f602871ff959f00 Author: Patrick Uiterwijk puiterw...@redhat.com Date: Fri Apr 17 11:29:48 2015 + Block the Riddler.io bot from accessing bodhi This bot does NOT follow robots.txt, even though it announces that it does, and it hits Bodhi so much that it pulls the bodhi servers under. An email has been sent to the maintainer of the bot, but for the time being, let's block it from using any bodhi resources. Signed-off-by: Patrick Uiterwijk puiterw...@redhat.com diff --git a/roles/bodhi/base/files/bodhi-app.conf b/roles/bodhi/base/files/bodhi-app.conf index 3e10a59..3f44964 100644 - --- a/roles/bodhi/base/files/bodhi-app.conf +++ b/roles/bodhi/base/files/bodhi-app.conf @@ -13,6 +13,8 @@ Alias /updates/tg_widgets/tgmochikit/packed/MochiKit/MochiKit.js /usr/lib/python Directory /usr/share/bodhi WSGIProcessGroup bodhi - - Order deny,allow + SetEnvIf User-Agent Riddler GoAway=1 + Order allow,deny Allow from all + Deny from env=GoAway /Directory -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBCgAGBQJVMPBRAAoJEIZXmA2atR5QBUAP/3+GOLJKRJ3hPB+GBBnx9Dg/ KDEdK6gCtOZUP0sYOPhEVmwTSHXjz/Z51Gvj9EO8FIFQFtnivA80tS2hiSp4x+gR K35z6cxIGbtlG7INLIBoyqTOD6bzvRe0mUP2cKL0+bgFxTzwbZx9HUrQd3PGwA4N 2W2SXROoVv2FVB4rpHTpQouY2VYdcyCDBsibjs2Y37Cxah/ltQC2FHaGlEMJSVCx DAQMAkxdGE0ww2AzRXysHDs46mHtpVFIOHgvtjd6A4NfMuB3gEGaJbcOj28AGmlt RIGU/arVp3OaNFwjAHyrHRCP56i68c7X/N0NwA/hWXLgOm3AJ4AbFoPPNSC0IwYo DVXvJBwRZp5hiuBED0T6/A6j2gVE52n1ytIcAmMznnw7qiupNxKLLpwow9T0RbYi 468aG97vvrFhr7Zj23i/yYsTXJvaMygHlG+nH82k/pdd93tXc5XVLL6MxjKAO53v gaHuvPcHhQuqXus/vv6nNXf/fRhtH3DlnTQRxMz5GZlXVdE9UCaROKNFcISgkCPR 62RYEBG4ZPPM6T9/Ps7CUtSVaCJnfZe1CSUhDlyA3JeCLizmLlAwUJXh3ysS1ZwX iGGkNjj/EdoJZdXz2941LurG2Q03o7cbcqGpZofJvRFDGdFhRdD5ePtqq1+/E0uw A0lPKspP13sayVSaqdBQ =QgK+ -END PGP SIGNATURE- ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
