So, I'm finishing up pre-deployment testing and I seem to have run into
limits of the system. I'm running htdig on a 256MB PIII, Mandrake 7.2
system. When I just index our own sites, digging is fast and the system
seems quite responsive. But, ideally, I'd like to dig 40-60 sites per topic
(say, Lotus Domino sites) and then maybe 3 or more topics. But it seems
that although this box has a large amount of RAM (it maxes at 384M) and a
40GB disk, the digging process is just too memory intensive and eveything
slows down to a crawl.
So, here's question: can I index large sites (like, say, lotus.com)? Or are
we just going to run into machine limits and I'm best off using htdig for
my own sites and leave the dream of indexing outside sites to a later project?
If I'm missing something, or their's an ideal configuration for attempting
this approach, please enlighten me.
Thanks!
-- DG
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>
