I am assuming that your generate.max.count property value is set to the default -1? Have you tried configuring more, smaller batchId's (fetch lists)? I don't have an immediate answer as to why overall, the FetcherJob is taking this amount of time and resources
On Wednesday, February 20, 2013, Roland <rol...@rvh-gmbh.de> wrote: > Hi Lewis, > > the GeneratorJob takes only ~5 minutes. > I'm running it in standalone mode, like this: > ./bin/nutch fetch 1361367698-1708119958 -threads 40 > > It's configured to fetch & parse, but it makes no difference if it only fetches: > FetcherJob: starting > FetcherJob: batchId: 1361367698-1708119958 > FetcherJob: threads: 40 > FetcherJob: parsing: true > FetcherJob: resuming: false > FetcherJob : timelimit set for : -1 > > --Roland > > > Am 20.02.2013 19:44, schrieb Lewis John Mcgibbney: >> >> Hi Roland, >> >> You say you start a fetch run, does this mean the FetcherJob or >> GeneratorJob? What kind of settings do you run your zNutch server with? > -- *Lewis*
