[SLUG] htdig or other searchers

2005-07-02 Thread David
I've just attempted to rebuild an htdig database after an extensive rebuild of a web site. htdig is only finding 20 documents, although there are actually about 300 linked html documents in the site. Does htdig go to the server (thereby parsing includes?). The site navigates perfectly from a b

Re: [SLUG] htdig or other searchers

2005-07-04 Thread telford
On Sun, Jul 03, 2005 at 04:58:10PM +1000, David wrote: > Does htdig go to the server (thereby parsing includes?). The site > navigates perfectly from a browser. Yes, in my experience it does, there are entries in the access_log. Beware if you are running virtual servers because usually it connec

Re: [SLUG] htdig or other searchers

2005-07-04 Thread David
On Tue, Jul 05, 2005 at 07:55:20AM +1000, [EMAIL PROTECTED] wrote: > On Sun, Jul 03, 2005 at 04:58:10PM +1000, David wrote: > > > Does htdig go to the server (thereby parsing includes?). The site > > navigates perfectly from a browser. > > Yes, in my experience it does, there are entries in the