At 7:06 PM -0500 9/1/99, Frank Martini wrote:
>that the skills I learn in HTDig _could_ at least get us part way to our
>goal. It does sound like a beefy dedicated machine is in order.
I don't think you need to worry about it not being able to do it.
IMHO (yes, I'm biased), there's very little separating ht://Dig from
a commercial package technically. Granted, it doesn't have a nice
clean admin interface, but if you just want to set something up and
leave it alone, you'll likely never notice.
Also, if you're someone who's shelling out a few thousand for
software, you're probably not worrying about getting a dedicated
machine either.
>Does this apply only for the first pass indexing of new sites?
No. If you have a large start_url attribute, it will grab a ton of
memory up front. It's all based on the size of the queue of URLs to
be indexed. Normally, the queue stays small since each page will add
a few (and subtract one). With the situation mentioned, right off the
bat, it builds a long queue.
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word unsubscribe in
the SUBJECT of the message.
