Recrawling on existing crawl directory using force option
---------------------------------------------------------
Key: NUTCH-601
URL: https://issues.apache.org/jira/browse/NUTCH-601
Project: Nutch
Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: Susam Pal
Priority: Minor
Added a '-force' option to the 'bin/nutch crawl' command line. With this
option, one can crawl and recrawl in the following manner:
{code}
bin/nutch crawl urls -dir crawl -depth 2 -topN 10 -threads 5
bin/nutch crawl urls -dir crawl -depth 2 -topN 10 -threads 5 -force
{code}
This option can be used for the first crawl too:
{code}
bin/nutch crawl urls -dir crawl -depth 2 -topN 10 -threads 5 -force
bin/nutch crawl urls -dir crawl -depth 2 -topN 10 -threads 5 -force
{code}
If one tries to crawl without the -force option when the crawl directory
already exists, he/she finds a small warning along with the error message:
{code}
# bin/nutch crawl urls -dir crawl -depth 2 -topN 10 -threads 5
Exception in thread "main" java.lang.RuntimeException: crawl already
exists. Add -force option to recrawl.
at org.apache.nutch.crawl.Crawl.main(Crawl.java:89)
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
