At 2:54 AM -0500 11/25/99, James A. Treacy wrote:
>Thanks for the response. It appeared that development on ht://Dig had
>stalled (this was well over a year ago) so I didn't reevaluate it.
Yes, development did stall about that time. Fortunately, thanks to
open source, it's picked up again. :-)
>My biggest concern is still speed. The Debian mailing list archives
>contain over 160000 files (> 1.6GB). The last time I tried indexing
>this was with 3.0.8b2. Do you think it can handle this? I'm guessing
>the most efficient way to handle updates is to reindex the current
>month each day, and then merge that with the index for everything
>before that.
This will be the most efficient method by far--it's simply an issue
of numbers. Obviously turning on local indexing helps too. It can
certainly handle 160,000 files without much problem. I know of two
sites using it for similar purposes, but with about 600,000 to more
than 1.5 million files.
>Another requirement I should have included in the list is the ability
>to restrict searches to files with a certain extensions. This is
No problem. The form allows you to restrict or exclude based on
strings in the URLs. So you can also restrict the search to certain
areas of the website easily.
>I'm downloading htdig-3.2.0-dev-112199.tar.gz right now and will take
>a look at it soon (probably within a week).
I'd actually suggest trying 3.1.2 or 3.1.3. While we're moving
towards a 3.2.0b1 release, the speed of the development tree is not
very good. We simply haven't made much of an attempt to optimize yet.
However, if phrase searching or regex searching is very important
then this is what you'll want to try.
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You'll receive a message confirming the unsubscription.
