Hi Rohan, You might also try the LiveHTTPHeaders firefox extension, it is also very good for this type of reverse engineering.
http://livehttpheaders.mozdev.org/index.html HTH, Marty Luke Paireepinart wrote: > Kent Johnson wrote: >> Rohan Deshpande wrote: >> >>> Hi All, >>> >>> the previous thread on screen scraping got me thinking of starting a >>> similar project. However, the problem is I have no idea what the POST >>> request is as there is no escape string after the URL when the resulting >>> page comes up. I essentially need to pull the HTML from a page that is >>> generated on a users machine and pipe it into a python script. How >>> should I go about doing this? Is it possible/feasible to decipher the >>> POST request and get the HTML, or use some screen scraping python libs a >>> la the javascript DOM hacks? I was thinking of the possibilities of the >>> former, but the interaction on the site is such that the user enters a >>> username/password and goes through a couple links before getting to the >>> page I need. Perhaps Python can use the session cookie and then pull >>> the right page? >>> > Have you tried using Firebug? It's an extension for Firefox. > You might be able to run it while you're navigating the site, and see > the communciation between you and the server and get the POST that way, > but I'm not completely certain about that. > -Luke > _______________________________________________ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor