Sanjay Arora <[EMAIL PROTECTED]> writes: > We are looking to select the language & toolset more suitable for a > project that requires getting data from several web-sites in real- > time....html parsing/scraping. It would require full emulation of the > browser, including handling cookies, automated logins & following > multiple web-link paths. Multiple threading would be a plus but not > requirement.
Believe it or not, everything you ask for can be done by Python out of the box. But there are limitations. For one, the HTML parsing module that comes with Python doesn't handle invalid HTML very well. Thanks to Netscape, invalid HTML is the rule rather than the exception on the web. So you probably want to use a third party module for that. I use BeautifulSoup, which handles XML, HTML, has a *lovely* API (going from BeautifulSoup to DOM is always a major dissapointment), and works well with broken X/HTML. That sufficient for my needs, but I haven't been asked to do a lot of automated form filling, so the facilities in the standard library work for me. There are third party tools to help with that. I'm sure someone willsuggest them. > Can you suggest solutions for python? Pros & Cons using Perl vs. Python? > Why Python? Because it's beautiful. Seriously, Python code is very readable, by design. Of course, some of the features that make that happen drive some people crazy. If you're one of them, then Python isn't the language for you. <mike -- Mike Meyer <[EMAIL PROTECTED]> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information. -- http://mail.python.org/mailman/listinfo/python-list