Georg Brandl ge...@python.org added the comment:
Thanks for the patch, fixed in r83238.
--
nosy: +georg.brandl
resolution: - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4108
mARK python.mblo...@xoxy.net added the comment:
this looks like a good fix. i've put it into my own copy.
--
nosy: +mbloore
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4108
___
New submission from taskinoor hasan sajid [EMAIL PROTECTED]:
Check the robots.txt file from mathworld.
-- http://mathworld.wolfram.com/robots.txt
It contains 2 User-Agent: * lines.
From http://www.robotstxt.org/norobots-rfc.txt
These name tokens are used in User-agent lines in /robots.txt to
