I'm using a long running production cluster so I don't think the machine configuration is the issue, and if so, I'd expect it in the fetch phase, wouldn't you ? On Dec 1, 2013 9:41 PM, "S.L" <[email protected]> wrote:
> I was able to execute a crawl of couple of hundred thousand URLs in local > mode , I did not get any OOM exceptions , what machine configuration do > you use ? > > > On Sat, Nov 30, 2013 at 4:43 PM, Amit Sela <[email protected]> wrote: > > > I get OOM exception in parse phase. > > I think it's related to https://issues.apache.org/jira/browse/NUTCH-1640 > > Did anyone succeed in fetching and parsing hundreds of thousands or even > > millions of pages with Nutch 1.7 ? > > >
