Hi,
I apologize if this is in the FAQ. We have here a request from one of the web
sites we index to not index the PDF files. I can't do that at the global level
because I want to continue indexing the other sites and in general we do not support
individual request. It would be more simple if the webmaster of this site could just
use a robot.txt like this:
User-agent: htdig-udem
Disallow: *.pdf
Disallow: *.PDF
Is this possible with ht://Dig 3.1.6?
Thanks.
-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
