On Mon, 15 Jul 2002, Alexander F Avdonkin wrote: > Can you tell your hardware configuration, parameters "WordCacheSize" , > "HrefCacheSize", "DeltaBufferSize" from "aspseek.conf" and MySQL config ?
Additionally, how many servers are you indexing? If your only indexing 10 but running 500 threads you'll have 490 threads doing nothing at any given time. Also check value of MinDelay (but first read the "Guidelines for Robot Writers" at http://www.robotstxt.org/wc/guidelines.html under section "Don't hog resources" - it's not considered polite to index too fast). Of course, if all the servers your indexing are managed by you then you might consider adjusting MinDelay to "0s". > Wim Godden wrote: > > > What other ways are there to speed up indexing ? We have a connection > > that's capable of getting 300-400kbyte/sec all the time, but even with > > 500 threads, indexing won't use more than 30-40kbyte/sec. The database > > server copes easily and the load on the indexer is only 40%. > > Why isn't it going any faster ??
