Re: [pydotorg-www] robots.txt make web archive inaccessible

2014-06-25 Thread anatoly techtonik
On Wed, Jun 25, 2014 at 11:13 AM, M.-A. Lemburg wrote: > On 25.06.2014 06:16, anatoly techtonik wrote: >> http://web.archive.org/web/*/http://docs.python.org/release/2.2.3/whatsnew/node5.html > > Can you propose an alternative robots.txt file ? > > """ > # Prevent development and old documentation

Re: [pydotorg-www] robots.txt make web archive inaccessible

2014-06-25 Thread M.-A. Lemburg
On 25.06.2014 06:16, anatoly techtonik wrote: > http://web.archive.org/web/*/http://docs.python.org/release/2.2.3/whatsnew/node5.html Can you propose an alternative robots.txt file ? """ # Prevent development and old documentation from showing up in search results. User-agent: * # Disallow: /dev

[pydotorg-www] robots.txt make web archive inaccessible

2014-06-24 Thread anatoly techtonik
http://web.archive.org/web/*/http://docs.python.org/release/2.2.3/whatsnew/node5.html -- anatoly t. ___ pydotorg-www mailing list pydotorg-www@python.org https://mail.python.org/mailman/listinfo/pydotorg-www