Kir Kolyshkin wrote: > > Well, even that won't do... I tried running up to 500 threads, but that simply >slowed > > everything down even more. MySQL is on a seperate box and s.cgi is not required >yet, > > because I'm still indexing and not providing search access yet. > > Please describe your setup - network connection, server RAM and disks, > MySQL's key_buffer size, what do you index, etc.
100Mbit/sec switched network MySQL database server has 1GB RAM and Seagate 40GB 7200rpm disk, key_buffer = 256MByte, indexing the .be domain Indexer is a P3-1Ghz with 512MB RAM But my current setup (using own PHP-scripts) : 8 x P3-1Ghz, retrieving pages 5 x P3-1Ghz, retrieving links from pages (including javascript links) 5 x P3-1Ghz, indexing 1 x P3-1Ghz MySQL server Total speed : about 80 pages per second :-))) Greetings, Wim -- ------ 11 EURO (incl. BTW) voor een .be domein ! Tijdelijk aanbod tot 1 september ! Snel naar http://domain.firstlinknetworks.com ! -- Adverteren.be - 100% Nederlandstalig adverteren op kwalitatief hoogstaande sites !
