>limited to this. I can tell you I have aspseek running on a >dedicated point-to-point T-1 and I'm able to index approximately one million >URLs a day many of which are on different servers. Aspseek.conf is >basically set at the default settings.
Yup, I've got a full point to point T1 dedicated to this task. >directive in aspseek. Even setting this at 50% of the total >bandwidth I have available, index will still consume the entire 1.544MB. What I found was that each time I started the process, it would jump right up and use everything it could. But after a while, it always slowed down. Cutting back on the (-N) number of processes helped this out also. >If you are using an external DNS server then >this will certainly slow things down. From your previous post saying it >was running for days at a time for only 10,000+ URLs sounds like your >resources are limited. The DNS is on another server on the backbone which is local and so not remote or anything but also not on the same system either. I'm not sure what happened with the initial indexing, it just died and I didn't find any reason. Seems to be doing fine now, it's always got some 1.3 million URL's in queue. But, after a week or so, I'm just wondering now if this is going to take another week, a month, what? Mike
