Freeze break request: block search engines from packages

2015-09-16 Thread Patrick Uiterwijk 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi,

As some of you might have noticed, we are currently getting a lot of requests
sent to packages, which is caused by search engines.
Since engines don't have any use in packages since it's just an aggregator,
I would like to disallow their access.

Can I get +1s on the following patch?


- -- 
With kind regards,
Patrick Uiterwijk
Fedora Infra



commit 25f71933abb4c571948454fc7507d190e1b34b06
Author: Patrick Uiterwijk 
Date:   Mon Sep 14 12:03:20 2015 +

Robots have no use in fedoracommunity as its just an aggregator

Signed-off-by: Patrick Uiterwijk 

diff --git a/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org 
b/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org
new file mode 100644
index 000..31e3660
- --- /dev/null
+++ b/roles/httpd/website/files/robots/robots.txt.apps.fedoraproject.org
@@ -0,0 +1,3 @@
+User-agent: *
+Disallow: /packages/
+Crawl-delay: 1

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBCgAGBQJV+VgcAAoJEIZXmA2atR5Q5B4P/0EtJYIiuVjYIa6Fo+6izonn
3K0lWkLmmILzQLm1p2QmNFMtg5t2rL6tWT1uKJZFuPvFybmIKjihTiHQwK/SED4t
iG4w4lRQYfV8+ejmhbzeDb6FV0kw0dggdCgsD1GUFrUGasfa7hkc97+J/L/M7vBV
v1Tayrwr+fvZlrWwyhslhCnilmhyfCRux9zjeXu0X3NbmwDKAO8gYlnfeC0vN+ul
Q1YcAd+b+9pVYBrWm574g7iPtg03SWxdkeY/21rXpbxyI51KaV4gGfCE7jwC6b6F
V7fh7R/PA0v2wyEIpi2SeuDoBLH5Bskqanri3i+84aHJ6GneGHw2nDPGc+gN6fXu
RQ/d1QJl+7q7Gj04ACpwysUVCI4iqHi0X8M8Ev9vkGD6G24K/GQNSRIT/Ft6jbN+
KjzD8KQe6QBhi5dSeFEcoltOt+MphEEzbF1nJdMXcj3f0Ig46p0Blzkz3XEL6/VZ
FvULeNYY+5dobRq4AM2LfwoioxYFSkP1L6TD16to+RGEw/8f7Z70H9oo4RD/xk8W
tJ2ttomxWNem4ApZ3tdCgq9b2IEQ8KG8FaWQX/e8mkfUtR2tstjJZPBwE4pFrScv
9rhoAF+Pugjw7vnsWj5p60qC//GcPBzvJ0OHZkkkZV10RuOwjWJCG4Vv2e4d04HF
pgKkEy1OA/L3SC6SvlGt
=uU4b
-END PGP SIGNATURE-
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/infrastructure@lists.fedoraproject.org

Re: Freeze break request: block search engines from packages

2015-09-16 Thread Ralph Bean 
On Wed, Sep 16, 2015 at 01:53:00PM +0200, Patrick Uiterwijk wrote:
> Hi,
> 
> As some of you might have noticed, we are currently getting a lot of requests
> sent to packages, which is caused by search engines.
> Since engines don't have any use in packages since it's just an aggregator,
> I would like to disallow their access.
> 
> Can I get +1s on the following patch?

Looks good to me.  +1


signature.asc
Description: PGP signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/infrastructure@lists.fedoraproject.org

Re: Freeze break request: block search engines from packages

2015-09-16 Thread Kevin Fenzi 
+1 from me. 

kevin


pgpYtK6_W7m43.pgp
Description: OpenPGP digital signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/infrastructure@lists.fedoraproject.org