Re: [vox-tech] Perl question - How do I use an HTTP header to download a file [SOLVED]?

2005-03-02 Thread Jay Strauss
Jay Strauss wrote:
Hi,
Using LWP::UserAgent and its pals, I get back the response (below via 
Data::Dumper).  If I use the same $request object and feed it to 
www::mechanize it does the right thing and the data file is in the 
-content.  I can't see www::mech does it, or how to do it properly with 
 LWP::UserAgent

Basically the response is saying, go to this location with this cookie 
and download the file.  How am I supposed to use this header?  Am I 
suppose to feed this header into some other type of object and do a 
request on it?

Any suggestions?
Thanks
Jay
$VAR1 = bless( {
 '_protocol' = 'HTTP/1.1',
 '_content' = 'htmlheadtitleObject 
moved/title/headbody^M
h2Object moved to a 
href=\'/DelayedQuote/QuoteData.dat\'here/a./h2^M
/body/html^M
',
 '_rc' = '302',
 '_headers' = bless( {
'client-response-num' = 1,
'cache-control' = 'private',
'set-cookie' = [

'CBOESiteTrackingID=d58134de-4d88-4d13-9d5e-c831599506a6; expires=Tue, 
17-Feb-2105 12:17:19 GMT; path=/',

'WEBTRENDS_ID=; expires=Wed, 16-Feb-2005 12:17:19 GMT; path=/',
'DownLoadError=; expires=Wed, 16-Feb-2005 12:17:19 GMT; path=/',
'QueryData=79D55F9BFFCA04C80D64EFC35FDECCEC487BB6D44EF9493E2214453402DC21A172A00E496D6348D6C2B9CD6E60C0BC172ACE78326718524614957081569C6A566053D27798472A2A5F90F1CBC849A20ED2FDB94956EFC33E; 
path=/'
],
'location' = 
'/DelayedQuote/QuoteData.dat',
'date' = 'Thu, 17 Feb 2005 
12:17:19 GMT',
'client-peer' = 
'198.160.148.116:80',
'content-length' = '144',
'x-aspnet-version' = '1.1.4322',
'client-date' = 'Thu, 17 Feb 
2005 12:14:20 GMT',
'content-type' = 'text/html; 
charset=iso-8859-1',
'title' = 'Object moved',
'server' = 'Microsoft-IIS/6.0'
  }, 'HTTP::Headers' ),
 '_msg' = 'Found',
 '_request' = bless( {
'_content' = 
'__EVENTTARGET=__EVENTARGUMENT=NOVIEWSTATE=dDwtODc5MTM4MTUwOzs%2B6wDeQv0gEC342sUPKG%2B3n6ZStJ8%3DucHeader%3AucCBOEHeaderLinks%3AucCBOEHeaderSearch%3Asearchtext=txtTicker=cmdSubmit=Download', 

'_uri' = bless( do{\(my $o = 
'http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx')}, 
'URI::http' ),
'_headers' = bless( {

'user-agent' = 'Windows IE 6',
'content-type' = 'application/x-www-form-urlencoded',
'content-length' = 196
 }, 
'HTTP::Headers' ),
'_method' = 'POST'
  }, 'HTTP::Request' )
   }, 'HTTP::Response' );
___
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech


By adding the referer things worked out:
#!/usr/bin/perl
use WWW::Mechanize;
use LWP::UserAgent;
use HTML::Form;
use Data::Dumper;
$url= http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx;;
$agent  = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)';
$symbol = '';
$ua = LWP::UserAgent-new(cookie_jar={}, agent=$agent);
push @{ $ua-requests_redirectable }, 'POST';
$resp = $ua-get($url);
$form = HTML::Form-parse($resp-content,$resp-base);
$form-find_input('__VIEWSTATE')-name('NOVIEWSTATE');
$form-value(txtTicker = $symbol);
my $req = $form-click;
$req-header('referer' = $url);
my $resp = $ua-request($req);
print $resp-content;

Jay
___
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech


[vox-tech] Perl question - How do I use an HTTP header to download a file?

2005-02-17 Thread Jay Strauss
Hi,
Using LWP::UserAgent and its pals, I get back the response (below via 
Data::Dumper).  If I use the same $request object and feed it to 
www::mechanize it does the right thing and the data file is in the 
-content.  I can't see www::mech does it, or how to do it properly with 
 LWP::UserAgent

Basically the response is saying, go to this location with this cookie 
and download the file.  How am I supposed to use this header?  Am I 
suppose to feed this header into some other type of object and do a 
request on it?

Any suggestions?
Thanks
Jay
$VAR1 = bless( {
 '_protocol' = 'HTTP/1.1',
 '_content' = 'htmlheadtitleObject 
moved/title/headbody^M
h2Object moved to a href=\'/DelayedQuote/QuoteData.dat\'here/a./h2^M
/body/html^M
',
 '_rc' = '302',
 '_headers' = bless( {
'client-response-num' = 1,
'cache-control' = 'private',
'set-cookie' = [

'CBOESiteTrackingID=d58134de-4d88-4d13-9d5e-c831599506a6; expires=Tue, 
17-Feb-2105 12:17:19 GMT; path=/',

'WEBTRENDS_ID=; expires=Wed, 16-Feb-2005 12:17:19 GMT; path=/',
'DownLoadError=; expires=Wed, 16-Feb-2005 12:17:19 GMT; path=/',
'QueryData=79D55F9BFFCA04C80D64EFC35FDECCEC487BB6D44EF9493E2214453402DC21A172A00E496D6348D6C2B9CD6E60C0BC172ACE78326718524614957081569C6A566053D27798472A2A5F90F1CBC849A20ED2FDB94956EFC33E; 
path=/'
],
'location' = 
'/DelayedQuote/QuoteData.dat',
'date' = 'Thu, 17 Feb 2005 
12:17:19 GMT',
'client-peer' = 
'198.160.148.116:80',
'content-length' = '144',
'x-aspnet-version' = '1.1.4322',
'client-date' = 'Thu, 17 Feb 
2005 12:14:20 GMT',
'content-type' = 'text/html; 
charset=iso-8859-1',
'title' = 'Object moved',
'server' = 'Microsoft-IIS/6.0'
  }, 'HTTP::Headers' ),
 '_msg' = 'Found',
 '_request' = bless( {
'_content' = 
'__EVENTTARGET=__EVENTARGUMENT=NOVIEWSTATE=dDwtODc5MTM4MTUwOzs%2B6wDeQv0gEC342sUPKG%2B3n6ZStJ8%3DucHeader%3AucCBOEHeaderLinks%3AucCBOEHeaderSearch%3Asearchtext=txtTicker=cmdSubmit=Download',
'_uri' = bless( do{\(my $o = 
'http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx')}, 'URI::http' ),
'_headers' = bless( {

'user-agent' = 'Windows IE 6',
'content-type' = 'application/x-www-form-urlencoded',
'content-length' = 196
 }, 
'HTTP::Headers' ),
'_method' = 'POST'
  }, 'HTTP::Request' )
   }, 'HTTP::Response' );
___
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech