Klaus,
Thanks for the info!
If we can get enough developers interested I've got an idea to dumb the current db code and move to using Lucene/Clucene. This should scale much better and remove a large chunk of code from active maintenance.
On Sun, 12 Dec 2004, minimoa wrote:
I have been using htdig for years to index my local pc. htdig is part of the Suse linux distribution I use. On a 2.4GHz cpu indexing the entire pc took 2 hours with htdig 3.1.6, but with version 3.2.0b6 it takes much longer. The words database becomes htdig's bottleneck when it exceeds 200MB on my pc. So I divided the search domain according to local directory structure and got smaller words databases and faster digs. The disadvantage is a lot of config files.
Searching itself is faster: 3000 hits for FAQ are found in 30 secs by 3.2.0b6 while 3.1.6 needed 60 secs. And phrases are a quantum leap.
Currently merging of databases is not any faster than indexing directly. So if database update cannot be accelerated due to limitations of db software, techniques for divide and conquer could be improved and should be made more common in documentation and configuration examples of htdig.
Thanks for a great software Klaus
--
Neal Richter Knowledgebase Developer
RightNow Technologies, Inc.
Customer Service for Every Web Site
Office: 406-522-1485
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/
_______________________________________________
ht://Dig Developer mailing list:
[EMAIL PROTECTED]
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-dev
