Re: crawling a list of urls

Hi C.B., This is way to vague. We really require more information regarding roughly what kind of results you wish to get. It would be a near impossible task for anyone to try and specify a solution to this open ended question. Please elaborate Thank you On Thu, Jul 7, 2011 at 12:56 PM, Cam

Re: crawling a list of urls

Hello Lewis, Pardon me for the non-verbose desription. I have a set of urls, namely product urls, in range of millions. So I want to write my urls, in a flat file, and have nutch crawl them to depth = 1 However, I might remove url's from this list, or add new ones. I also would like nutch to

Re: crawling a list of urls

See comments below On Thu, Jul 7, 2011 at 4:31 PM, Cam Bazz camb...@gmail.com wrote: Hello Lewis, Pardon me for the non-verbose desription. I have a set of urls, namely product urls, in range of millions. Firstly, (this is juts a suggestion) but I assume that you wish Nutch to fetch the

Re: crawling a list of urls

Thank you Lewis, this has been very illustrative, especially about deleting documents. Best. On Thu, Jul 7, 2011 at 6:51 PM, lewis john mcgibbney lewis.mcgibb...@gmail.com wrote: See comments below On Thu, Jul 7, 2011 at 4:31 PM, Cam Bazz camb...@gmail.com wrote: Hello Lewis, Pardon me