I remember seeing those in the logs, but it's been a while. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ---- > From: caezar <[email protected]> > To: [email protected] > Sent: Friday, June 26, 2009 3:50:39 AM > Subject: Re: Nutch fetch performance > > > I don't think that they are waiting for 30 seconds. Http timeout is 5 > seconds. Number of retries is 2. Delay between retries is 0.5 seconds. So it > should be at most 10.5 seconds. > But however, is there a way to see the number of active fetcher threads? > > Ken Krugler wrote: > > > > The real question is how many active fetches you have running > > simultaneously. If most fetcher threads are idle, waiting for 30 > > seconds to pass before fetching the next page from a host, then 10 > > pages/second might be an expected fetch rate. > > > > -- > View this message in context: > http://www.nabble.com/Nutch-fetch-performance-tp24203861p24216312.html > Sent from the Nutch - User mailing list archive at Nabble.com.
