Re: (Fwd) Automatic posting to forms

[Daniel Stenberg]

On Fri, 8 Mar 2002, Tony Lewis [EMAIL PROTECTED] wrote:

 As promised in my earlier note, there is a second feature I'm looking for
 in wget. This feature is the ability to automatically post to forms. I'm
 thinking of something along the lines of a command line argument like:

   --auto-post=FILE

 where FILE would contain data such as:

 form=/cgi-bin/auth.cgi
 name=id value=tony
 name=pw value=password

 With this information, any time that wget encounters a form whose action is
 /cgi-bin/auth.cgi, it will enqueue the submission of the form using the
 values provided for the fields id and pw.

Now, why would wget do this?

wget could very well just download that page for you, you write the script
that parses the page to figure out which fields to fill in and then you
construct the post data and then you POST using a tool that supports it.

I really can't see any reason why wget would do everything this internally.

 Is there anything else that seems obvious that I'm overlooking? Any
 cautions about the sections of code I'll be working with?

Yes, probably: when the form tags contains enctype='multipart/form-data'
you need to build an entirely different data stream (RFC1867 is the key
here).

I wrote a basic find the form info script once. It is far from perfect, but
it does the job on most pages: http://curl.haxx.se/programs/formfind.txt

I'd also like to point out that curl already supports both regular HTTP POST
as well as multipart formposts.

-- 
  Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77
   ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol

Re: (Fwd) Automatic posting to forms

[Tony Lewis]

Daniel Stenberg responded to my original suggestion:

  With this information, any time that wget encounters a form whose action
is
  /cgi-bin/auth.cgi, it will enqueue the submission of the form using
the
  values provided for the fields id and pw.

 Now, why would wget do this?

There are many examples of sites that require the user to post a form to
access other parts of the site -- sometimes the post contains user-supplied
data and sometimes it doesn't. If one wants to grab everything on the other
side of that form, having wget post the form seems like the way to get
there.

 Yes, probably: when the form tags contains enctype='multipart/form-data'
 you need to build an entirely different data stream (RFC1867 is the key
 here).

You're right. I had not yet thought about that flavor of posting.

 I'd also like to point out that curl already supports both regular HTTP
POST
 as well as multipart formposts.

Unless I'm misreading the curl manual, that only allows me to get one page.
However, I've never been inclined to invent wheels when I can download them.
I will study the curl source code related to posting before I wander too far
down this path.

Tony