Erno, www.mail-archive.com does this by putting HTML meta tags on index pages that instruct robots not to archive the page. This assumes you have control over the HTML pages. Jeff > search http://www.foo.org/mailing-list/* > not return as search results as urls that match */date.html, or > */thread.html, or end in '/'. > > but it should still follow the links in those files and index > the linked-to files (mailing list messages). ---------------------------------------------------------------------- To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] containing the single word "unsubscribe" in the body of the message.
