On Mon, 28 Oct 2024 at 16:06, Ridwan Naibi <[email protected]> wrote:
> Hello everyone, > > Can I please know an AWS service that I can use to crawl the entire web? > It was suggested to me as an easy replacement nutch. It seems to be that > answer was given to me but I can't find it in the email thread. Thanks. If you really mean to crawl the entire web, it would make sense to start by looking at https://commoncrawl.org/ Regards, Gora
