Greets,

I've written an industrial-strength search engine library for Perl (KinoSearch), and now I have clients who want me to work on a large- scale spidering app for them. Sort of like Nutch for Perl (<http:// lucene.apache.org/nutch>). Putch. :)

What efforts have already been undertaken in this area? A survey of existing CPAN releases that I should study would be great. I've written a small-scale spider using LWP::RobotUA. I've scanned over the WWW::Mechanize docs, but don't yet grasp its full capabilities. What else?

Thanks,

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

Reply via email to