Re: Crawling authenticated websites !

2010-03-18 Thread Susam Pal
On Thu, Mar 18, 2010 at 7:27 PM, Ranganath Cuddapah wrote: > Hello, > Is there a way to configure Nutch to crawl "forms authenticated" websites? > What I mean is the kind of websites which look up a database for > authentication/authorization and does not allow you to view secure pages > unless au

Crawling authenticated websites !

2010-03-18 Thread Ranganath Cuddapah
Hello, Is there a way to configure Nutch to crawl "forms authenticated" websites? What I mean is the kind of websites which look up a database for authentication/authorization and does not allow you to view secure pages unless authenticated. This need not be specifically on https, but on http too.