Which version of nutch are you using for this crawl ? (please specify this info whenever you are asking for help over @user. It would help us understanding the issue)
In your "bin/nutch crawl" commands, the "crawldb" ain't specified. So each invocation will produce a fresh "crawldb" using the timestamp when the command was invoked. If the seed urls, configs and params are not changed in between, there should be no difference in those 2 executions. This is how things are done in 1.x. I am not sure how its done in 2.x. Depending on your other mail thread over @user, I think that you are using 2.x. On Wed, Mar 20, 2013 at 3:59 PM, kamaci <furkankam...@gmail.com> wrote: > First I used that command: > > bin/nutch crawl /usr/apache-nutch-2.1/urls -depth 3 > > than I pressed ctrl + c and fetching has ended as usual. > > After that I used that command: > > bin/nutch crawl /usr/apache-nutch-2.1/urls -depth 2 -topN 2 > > But fetching takes long time than usual. So I think it continues to > previous fetching is that true? > > > 2013/3/21 Markus Jelsma-2 [via Lucene] < > ml-node+s472066n4049571...@n3.nabble.com> > > > If Nutch exits with an error then the segment is bad, a failing thread is > > not an error that leads to a failed segments. This means the segment is > > properly fetched but just that some records failed. Those records will be > > eligible for refetch. > > > > Assuming you use the crawl command, the updatedb command will be > > successful so there should be no issue here. What's the problem? > > > > > > -----Original message----- > > > > > From:kamaci <[hidden email]< > http://user/SendEmail.jtp?type=node&node=4049571&i=0>> > > > > > Sent: Wed 20-Mar-2013 23:48 > > > To: [hidden email]< > http://user/SendEmail.jtp?type=node&node=4049571&i=1> > > > Subject: How to Continue to Crawl with Nutch Even An Error Occurs? > > > > > > When I crawl with Nutch and error occurs (i.e. when one of threads > > doesn't > > > come within a time) it stops crawling and exits. > > > > > > Is there any configuration to continue crawling even a such kind of > > error > > > occurs at Nutch? > > > > > > > > > > > > -- > > > View this message in context: > > > http://lucene.472066.n3.nabble.com/How-to-Continue-to-Crawl-with-Nutch-Even-An-Error-Occurs-tp4049567.html > > > Sent from the Nutch - User mailing list archive at Nabble.com. > > > > > > > > > ------------------------------ > > If you reply to this email, your message will be added to the discussion > > below: > > > > > http://lucene.472066.n3.nabble.com/How-to-Continue-to-Crawl-with-Nutch-Even-An-Error-Occurs-tp4049567p4049571.html > > To unsubscribe from How to Continue to Crawl with Nutch Even An Error > > Occurs?, click here< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4049567&code=ZnVya2Fua2FtYWNpQGdtYWlsLmNvbXw0MDQ5NTY3fDEyODM4MDc0Mg== > > > > . > > NAML< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml > > > > > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/How-to-Continue-to-Crawl-with-Nutch-Even-An-Error-Occurs-tp4049567p4049577.html > Sent from the Nutch - User mailing list archive at Nabble.com. >
