Would you be willing to share some more details on this PHP setup? What is the backend comprised of for spidering etc?
How many machines and is there a URL that we could check things out at? Just curious...:) Paul -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Wim Godden Sent: Monday, August 12, 2002 9:05 AM To: [EMAIL PROTECTED] Subject: Re: [aseek-users] More then 1 index server? Kir Kolyshkin wrote: > > I think he needs it for the same reason many of us would like that > > feature : one indexer is way too slow. If you want to index a whole > > ccTLD, it'll take you several months with aspseek. > > Hmm have you tried higher number after -N together with upgrading your > server to have more RAM and higher disk I/O throughput? Also, moving > MySQL to separate box, and searchd to another separate box helps a > lot. Actually s.cgi can be put on "yet another" box (I'm not sure if > this will help), so you will end up with four machines. Well, even that won't do... I tried running up to 500 threads, but that simply slowed everything down even more. MySQL is on a seperate box and s.cgi is not required yet, because I'm still indexing and not providing search access yet. > Also, PageRanks will be computed separately > for two indexes, which is not a good thing. Indeed... might as well tell your visitors the search results aren't good. Anyway, not a problem for me anymore, since I've stopped using aspseek and built my own system in PHP, so I can spread the load over our webserver farm... works like a charm ! Greetings, Wim -- ------ 11 EURO (incl. BTW) voor een .be domein ! Tijdelijk aanbod tot 1 september ! Snel naar http://domain.firstlinknetworks.com ! -- Adverteren.be - 100% Nederlandstalig adverteren op kwalitatief hoogstaande sites !
