Philippe 'BooK' Bruhat wrote:

Le mercredi 17 août 2005 à 06:44, Peter Scott écrivait:
On Mon, 15 Aug 2005 13:40:19 -0700, Johnc wrote:
Hi, I'm trying to write a bot to login to the scottrade trading site, and while I'm taking care to grok hidden fields and I've initalized a cookie jar for my LWP agent, posting the login info to the form still just returns the login form
again (no error message).
Use WWW::Mechanize and you won't have to grok hidden fields or cookies.

The ->click vs ->submit issue mentionned before is an important
trick. click() sends back a form parameter with the name of the button,
while submit() doesn't. Apparently, some sites expect to receive this
field too.

Are there any tutorials out there that address common caveats for this sort of thing? Or what else do I potentially need to take care of to make sure my LWP agent is getting all of the necessary session data to log in?
The big danger is JavaScript.  I have found no alternative to looking at
the source and analyzing the source by hand to see whether JavaScript
might be modifying any form parameters.  However so far this hasn't taken
me very long.

I've seen JavaScript used to add Cookies as well. The server doesn't
send any cookie, but the client send some back.

Since I do not like reading the JavaScript source, I use a simple
proxy based on HTTP::Proxy (found under eg/logger.pl in the HTTP::Proxy
distribution), which prints out interesting headers (Cookie, Location,
etc) and form fields. It's usually enough for finding out what the web
robot needs to send to the server.

Hi, yes I've discovered WWW::Mechanize, and it's just what I needed.  I
am able to login to the
scottrade site with just a submit() call.
I think the caveat about using click() will come in handy in the near
future, too.  Though I believe
just adding the button parameter to your submit() call may serve to
disambiguate in some cases as well.

Thanks,
-John

Reply via email to