Re: ignoring robots.txt

[Maciej W. Rozycki]

On Wed, 18 Jul 2007, Josh Williams wrote:

> Is there any particular reason we don't have an option to ignore robots.txt?

 There is no particular reason, so we do.

  Maciej