Geoff Hutchison wrote:
>
> At 10:14 AM +1100 12/19/00, Terry Collins wrote:
> >And make sure you don't ignore robots.txt
>
> Yes, though someone would need to alter the code to do this.
If you are doing an external site, it shouldn't be to much effort to
just read this and set the excludes.
Courtesy thing.
--
Terry Collins {:-)}}} Ph(02) 4627 2186 Fax(02) 4628 7861
email: [EMAIL PROTECTED] www: http://www.woa.com.au
WOA Computer Services <lan/wan, linux/unix, novell>
"People without trees are like fish without clean water"
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>
