Hi All,
Here is the problem: I need Nutch to crawl ONLY the URLs I've injected.
Currently, by setting db.ignore.external.links to true I've made Nutch not
to automatically crawl URLs found as external links from on crawled pages.
But it is still crawling URLs found as internal links (seems that
db.ignore.internal.links does not affects this). I don't want to create URL
filters, because there are millions of URLs, and not possible to write a
regexps for them. So is there a way to achieve this?
--
View this message in context:
http://www.nabble.com/How-to-tell-Nutch-to-crawl-ONLY-the-URLs-I%27ve-injected-tp24204304p24204304.html
Sent from the Nutch - User mailing list archive at Nabble.com.
