According to Geoff Hutchison:
>> that I get when I turn the debugging on. Is there any way in htdig to
>> overwrite this ?
>
>No. That would violate the robot exclusion standard.
>
>> Robots.txt line: User-agent: *
>> Found 'user-agent' line: *
>> Robots.txt line: Disallow: /
>
>This tells all robots to ignore all files on this server. So ht://Dig is
>doing exactly what it's told.
>
>My suggestion would be to have the server administrator change the
>robots.txt file.
ANother solution would be to tell ht://Dig to use an alternate
robots.txt file. See http://www.htdig.org/attrs.html#robotstxt_name
for more information.
hth,
Torsten
--
InWise - Wirtschaftlich-Wissenschaftlicher Internet Service GmbH
Waldhofstra�e 14 Tel: +49-4101-403605
D-25474 Ellerbek Fax: +49-4101-403606
E-Mail: [EMAIL PROTECTED] Internet: http://www.inwise.de
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word unsubscribe in
the SUBJECT of the message.
