According to [EMAIL PROTECTED]:
> I am running into some situations where HTDIG appears to be going into an
> infinite loop. The pattern is that it either does, or does not, occur when
> searching a specific website; re-attempting the HTDIG--against the same
> site--usually produces the same result.
>
> Is there some straightforward way to limit the number of bytes which can be
> written into either wordlist or the log file; to ensure unconditional
> termination of processing once this limit is exceeded? (In practice, HTDIG
> is invariably still producing output when it's externally terminated; I
> haven't run into any instances of looping without producing output).
There's a server_max_docs config attribute that will limit the number of
documents htdig will index per server. However, you may want to run htdig
with one or more -v options to get some feedback on what it is doing on the
servers that seem problematic. It might give you some hints as to how you
can tweak your exclude_urls and limit_urls_to attributes to avoid indexing
certain branches on these servers that you'd rather avoid.
See http://www.htdig.org/attrs.html for descriptions of all these attributes.
--
Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]>
Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba Phone: (204)789-3766
Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
