> >But the index pages are excluded from the database, so it never checks > >those for new links. > ...
> As far as keeping a list of excluded documents, this may be the right > way to go. Right now htmerge/htpurge completely remove all traces of > a document if it was marked "noindex." Probably the solution is to > leave the document (still marked "noindex") but make sure all words > are removed from the word db. This way it would never come up in a > search. As a workaround I added the threads.html files of my mailing list archives to the start urls in htdig's config files. These files are marked 'noindex' but adding them to the start urls makes the update digs look at them and from there find any new messages. Roman Maeder _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
