On Tue, 2 Sep 2003, Marc G. Fournier wrote: > > okay, first off ... what URL? I've been working on, and testing, archives > all week, and altho getting things down below a 2min search appears to be > near impossible (anyone know something *better* then mnogosearch that will > handle >250k URLs and growing ... ? we tried ht/Dig before, and the > resources it suck'd were outrageous) ...
Hi Marc, I use htdig at work to index a rather large web site and it doesn't use resources all that bad. My htdig db directory is only 250 megs and it screams. Are you sure it was that bad? Would it be ok if I pointed a test instance of htdig at postgresql.org at night and indexed it to get an idea of how much space it'll take to index it? I'm a huge fan of htdig/ warts and all, so I'd kinda like to see if it can be made to work. ---------------------------(end of broadcast)--------------------------- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly