Getting returned from values???

[Roderick A. Anderson]

Sorry to hit the list without a proper lurking period but this came on 
fast and I'm ducking flying materials and dodging around the muck.

We use an application (web based) that one page accepts two input fields
and generates a hidden field that is carried along with each new form and
page.  To hide some of the more gruesome details I'd like to do this in
the background so the returned value never shows up in the pages we see.
(The application providers/developers seem reluctant to share how they are
generating the returned value - I hate closed source software.)

Well I figured WWW:Mechanize was the ticket but I am now up against the
wall.  I can't figure out how to take the returned page and get just that
field's value.  All the examples I've found of using Mech are looking for
non-form information and though I could use those techniques I thought
there has to be an easier way.  Do I need to use CGI.pm (which I'm already
using in the script) and if so how do I get the Mech results into a CGI
instance?

As you'll notice I'm kind of rambling as I'm not sure of the right terms 
for what I'm trying to accomplish.


TIA,
Rod
-- 
  "Open Source Software - Sometimes you get more than you paid for..."

libwww-perl-5.70

[Gisle Aas]

I'm trying to go through my backlog of bug reports and patches for
LWP.  Today I've been fixing HTML::Form issues, and I've just uploaded
libwww-perl-5.70 as snapshot of this work-in-progress.  I expect more
releases to come out soon.

The changes that made it to v5.70 where:

File::Listing::apache by Slaven Rezic <[EMAIL PROTECTED]>

HEAD requests now work properly for ftp: URLs.
Patch by Ville Skyttä <[EMAIL PROTECTED]>.

LWP::UserAgent: The protocols_allowed() and protocols_forbidden()
methods are now case insensitive.  Patch by Ville Skyttä
<[EMAIL PROTECTED]>.

Avoid warning from HTTP::Date on certain invalid dates.
Patch by David Dick <[EMAIL PROTECTED]>.

HTML::Form::param() is an alternative interface for inspecting
and modifying the form values.  It resembles the interface
of CGI.

HTML::Form documentation updated.  Lots of typo fixes and improves
by Martin Thurn <[EMAIL PROTECTED]>.

HTML::Form will treat any unknown input types as text input.
This appears to be what most browsers do.

HTML::Form::parse() can now take an HTTP::Response object
as argument.

The "checkbox" and "option" inputs of HTML::Form can now be
turned on with the new check() method.

The inputs of HTML::Form can now track alternative value
names and allow values to be set by these names as well.
Currently this is only used for "option" inputs.

HTML::Form's dump() method now print the name of the form if
present.

file won't download.. what am i missing ?

[James.Q.L]

okay, here i am trying to download some realplayer type of files from www.rick.com
the file won't download even from regular browser. but it does play thr realplayer 
when i click
the file. 

it probably the case like other media files that users can't download but can 
linsten/watch
online. but anyway, i am just playing it with LWP.

here is my code:

#!/usr/bin/perl -w

use strict;
use HTTP::Request::Common qw(GET POST);
use LWP::UserAgent;
use HTTP::Cookies;

my $accept =  1);
#$ua->cookie_jar(HTTP::Cookies->new(file=>'mycookie',autosave=>1,ignore_discard => 1));
$ua->cookie_jar(HTTP::Cookies->new(file=>'mycookie',hide_cookie2=>1));
$ua->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windws NT 5.1)');
$ua->get($refer_url); # grab cookie from refer page

my $req = GET $ra_url;
$req->referer($refer_url);
$req->header('Accept-Language' => 'en-us');
$req->header('Accept' => $accept);
my $resp = $ua->request($req);

open F,">test.ra";
print F $resp->content;
close F;

the http traffic between them is suppose to be shown as 

GET /audio/interviews/danacarvey073002.ra HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword,
application/x-shockwave-flash, application/vnd.ms-excel, 
application/vnd.ms-powerpoint, */*
Referer: http://www.rick.com/interviews.html
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)
Host: www.rick.com
Connection: Keep-Alive
Cookie: popup_single_cookie=16;
poll_cookie_123=Should+suicide+during+a+live+concert+be+Televised+%3F

HTTP/1.1 200 OK
Date: Mon, 13 Oct 2003 06:47:53 GMT
Server: Apache/1.3.27 (Unix) PHP/3.0.18
Connection: close
Transfer-Encoding: chunked
Content-Type: audio/x-pn-realaudio

39 
http://www.rick.com/audio/interviews/danacarvey073002.ra

0

using code above i am getting 

GET /audio/interviews/danacarvey073002.ra HTTP/1.1
TE: deflate,gzip;q=0.3
Keep-Alive: 300
Connection: Keep-Alive, TE
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/msword,
application/x-shockwave-flash, application/vnd.ms-excel, 
application/vnd.ms-powerpoint, */* 
Accept-Language: en-us
Host: www.rick.com
Referer: http://www.rick.com/interviews.html
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windws NT 5.1)
Cookie: popup_single_cookie=16

HTTP/1.1 200 OK
Date: Mon, 13 Oct 2003 08:22:38 GMT
Server: Apache/1.3.27 (Unix) PHP/3.0.18
Connection: close
Transfer-Encoding: chunked
Content-Type: audio/x-pn-realaudio

39 
http://www.rick.com/audio/interviews/danacarvey073002.ra

0

the content of cookie file being loaded into HTTP::Cookies is 

#LWP-Cookies-1.0
Set-Cookie3: popup_single_cookie=15;
poll_cookie_123=Should+suicide+during+a+live+concert+be+Televised+%3F; path="/";
domain="www.rick.com"; path_spec; discard; version=0

The originaly content being saved from HTTP::Cookies is

#LWP-Cookies-1.0
Set-Cookie3: popup_single_cookie=1; path="/"; domain="www.rick.com"; path_spec; 
discard; version=0

note: it missed the 
poll_cookie_123=Should+suicide+during+a+live+concert+be+Televised+%3F; but
although i have the full entry in 'mycookie' file, the traffic is really looks like :

GET /audio/interviews/danacarvey073002.ra HTTP/1.1
#some more here
Referer: http://www.rick.com/interviews.html
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windws NT 5.1)
Cookie: popup_single_cookie=16  

where is the poll_cookie_123 part ?  
another weird things is that the value of popup_single_cookie is 16 as always. but if 
i have the
value setup as 16 in mycookie file, it will be shown in above as 17. so that's why i 
have 15
instead.


okay. just wondering if this ra file can actually be downloaded. 


regards,

Qiang

__
Do you Yahoo!?
The New Yahoo! Shopping - with improved product search
http://shopping.yahoo.com

Re: transfer-encoding buglet

[Gisle Aas]

Bob McElrath <[EMAIL PROTECTED]> writes:

> This version doesn't cause a warning if the "Connection" header isn't
> present.

Perhaps this warnings shows becase nothing sets 'http_connection'.  
Do you have further patches to Net::HTTP not shown here.


> Bob McElrath [EMAIL PROTECTED] wrote:
> > According to RFC 2616, section 3.6:
> > Whenever a transfer-coding is applied to a message-body, the set of
> > transfer-codings MUST include "chunked", unless the message is
> > terminated by closing the connection.
> > 
> > libwww currently does not correctly implement the "unless" clause where
> > the connection is closed but a transfer-coding header is present.

I agree.

> > Attached is a small patch which fixes this.  A URL that I've run
> > across that exhibit the problem with FilterProxy are:
> > 
> > http://larve.net/people/hugo/2000/07/ml-mutt

I don't see any problem with this one.  Do you have other examples?

Regards,
Gisle

> --- Methods.pm2003-10-05 12:21:21.0 -0700
> +++ /usr/share/perl5/Net/HTTP/Methods.pm  2003-10-05 13:01:53.0 -0700
> @@ -367,8 +367,12 @@
>   }
>   elsif (my $te = ${*$self}{'http_te'}) {
>   my @te = split(/\s*,\s*/, lc($te));
> +my @connection = split(/\s*,\s*/, 
> +defined ${*$self}{'http_connection'}
> +?${*$self}{'http_connection'}:"");
>   die "Chunked must be last Transfer-Encoding '$te'"
> - unless pop(@te) eq "chunked";
> + unless((pop(@te) eq "chunked") || 
> +(grep lc($_) eq "close", @connection));
>  
>   for (@te) {
>   if ($_ eq "deflate" && zlib_ok()) {