sandeep pujar wrote: > Greetings, > > Wanted to know if anybody had worked on form based > authentication for the nutch crawler. > > any pointers, suggestions would help. >
I have, without much success. Form-based authentication is different from site to site - most sites don't use just a plain form with username/password, but they use a wide variety of methods to check / protect the data being sent. In extreme cases forms will use an embedded challenge string, run a javascript-based md5 hash, and send only that ... in other cases some other tricks are played, with setting cookies, redirecting, running javascripts, etc. In the end only perhaps 1 out of 50 sites was using a plain form authentication, and even that with different field names on the form ... so I gave up. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
