All,

Does anyone have a magic number for the number of fetcher threads per host so 
that the site doesn't think they are under attack?  I'm trying to maximize the 
crawls without overwhelming some sites.

Thanks in advance,

Alex


      

Reply via email to