Alexander Aristov wrote:
Hi all I want to run two crawlers using single server at the same time across different seed lists. The question is Is it safe to use one binaries? I have developed scripts to specify different input/output locations but I wonder if nutch creates some temporarily folders during its work which I cannot control and so it would be possible situation when two crawlers overlap working data.
There aren't any conflicts in having multiple crawling jobs going and outputting to different directories at the same time. You do need to be careful about ordering if you are generating the crawl lists from a single crawldb and then updating back into that crawldb.
Dennis
Thanks Alexander Aristov
