Hello. I am new to Nutch. I need to use Nutch to index data into Solr.
Lets say I need to crawl some newspaper search pages and index any article regarding the word "java". I understand that I would need to point Nutch to the search result page. 1) What I need from nutch is not to crawl/index the result page which contains only the summary, but to follow each result link and index the content of the full article. 2) I would also need nutch to follow the pagination Next link to the next set of results and do the same as step 1) 3) Repeat 1) and 2) until there is no result left. Please, is this something that Nutch can easily do? Any hint will be well appreciated. Thanks. Arcadius.
