According to Wenge Fu:
> Hi Gilles,
>
> Happy 2001 !
>
> Thank you very much for your help. After I rebuilt the database, everything
> works fine. Should I run a cron job to updated the database using the "rundig.sh"
> you mentioned below? (I am not sure how this searching engine gets the latest
> information when some sites changed.) . I would appreciate your idea about the
> best solution for updating. Many thanks in advances.
>
> Best Regards!
>
> Wenge
Yes, you can run rundig.sh from a cron job to update the database
regularly. Just make sure you don't schedule it to run it so frequently
that it can start a job before the previous one finished, i.e. if
an update may take a full day, don't run an update every day because
there's a danger of an overlap occurring. If an update takes an hour
or a little more, there's no problem running it daily, but you can't run
it on an hourly basis. htdig has no lockout on the database, so if two
processes try to update the database simultaneously, you'll wind up with
an unusable database.
As for how htdig gets the latest information when some sites change, what
it does is it goes through every URL in the existing database and asks the
HTTP server for the document again, but using the "If-Modified-Since"
header in the request, so that it only fetches documents that have
been updated. If the HTTP server doesn't honour that header, and sends
the document anyway, htdig still checks the Last-Modified header that's
returned to see if the document has changed, and won't bother re-parsing
it if it hasn't. That's what makes update digs so much quicker than an
initial dig.
> > Gilles Detillieux wrote:
> > If rebuilding from scratch is too slow on your site, you'll want to look
>
> > into updating your existing databases rather than rebuilding them from
> > scratch each time. That means that instead of using the rundig script,
> > you should switch to an alternate one that does updates, such as the
> > one at:
> >
> > http://www.htdig.org/files/contrib/scripts/rundig.sh
> >
> > But, only do this when your existing database is solid and working
> > correctly. If the database shows any symptoms of corruption, you really
> > should rebuild it from scratch.
> >
> > > Gilles Detillieux wrote:
> > >
> > > > According to Wenge Fu:
> > > > > We use ht://Dig 3.1.4 on our web site
> > > > > (http://www.cadops.bnl.gov/Controls/) for doing internal search. It
> > > > > works well for a while. But recently, it doesn't work. When we do a
> > > > > search, it reports how many matches found, but doesn't display the
> > > > > results. Could you please tell me what could be the reason? Many
> > > > > thanks in advance.
> > > >
> > > > This is most likely due to a corrupted database, which you'd need to
> > > > rebuild from scratch, using "rundig" (or "htdig -i" then "htmerge").
> > > > You should also upgrade to version 3.1.5, as all earlier versions have
> > > > some pretty nasty bugs, especially a security problem in htsearch.
--
Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]>
Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba Phone: (204)789-3766
Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
