I'm crawling a static website on the same machine as Nutch (thus eliminating bandwith restrictions). stats: 99277 pages, 34 errors, 60.8 pages/s, 20410 kb/s
-----Original Message----- From: Ian Holsman [mailto:[EMAIL PROTECTED] Sent: Wed 6/20/2007 7:50 To: [email protected] Subject: how fast can nutch fetch urls ? I was wondering what kind of reqs/second people are achieving from a single node in a hadoop cluster in the fetch phase.
