In response to some off-line requests, here's an
example:

sub execute_request {

        my($req) = @_;
        
        $req->header('Accept-encoding' => 'gzip, deflate');
        
        print "\nReq
#$req_nbr.$sub_request:\n",$req->as_string;

        my $res = $ua->request($req);
        
        $last_response = $res;
        if ($res->is_success) {
                print "\nResp Headers #$req_nbr.$sub_request:\n";
                print $res->protocol, " ", $res->status_line, "\n";
                print $res->headers_as_string;
                my $follow_url = undef;  
                if ( $res->header('Content-encoding')) {
                        my $encoding = $res->header('Content-encoding');
                        if ($encoding =~ /deflate/o ) {
                                # process deflate encoding
                                print "\ndeflate encoding found\n";
                                $res->content(inflateBody($res->content_ref));
                        } elsif ($encoding =~ /gzip/o ) {
                                # process gzip encoding
                                print "\ngzip encoding found\n";
                                die "gzip decompression not yet implemented\n";
                        } else {
                                die "unsuported content-encoding $encoding returned
by browser\n";
                        }
                }
                if ( $res->header('Refresh') &&
$res->header('Refresh') =~ /url=(.*)/o ) {

                        print "\nFollowing re-direct via Refresh found in
HTTP headers or HTML <head>\n";
                        $follow_url = $1;

                } elsif ($res->content && $res->content =~
/document\.location\.href = \"(.+)\"/o ) {

                        print "\nFollowing re-direct via script
document.location.href = \n";
                        $follow_url = $1;

                } elsif ($res->content && $res->content =~
/<meta\s+http-equiv="?Refresh"?\s+content\s*=\s*"\d*;\s*[Uu][Rr][Ll]=(.*?)"/o) { 

                        print "\nFollowing re-direct via http-equiv=Refresh
found in HTML <body> or compressed <head>\n";
                        $follow_url = $1;
                }
                if ($follow_url) {
                        decode_entities($follow_url);
                        my $base =
$req->uri->scheme."://".$req->uri->authority;
                        #print "\nbase=$base\n";
                        $sub_request++;

                        get_url(URI->new_abs($follow_url,$base));       
                }
        } else {
                 print "\nFailed: ", $res->status_line;
        }
        return $res;    
}







> --- David Carter <[EMAIL PROTECTED]> wrote:
> > First, you need to determine exactly how the
> > redirect is being done. Look at
> > the response you are receiving, and determine what
> > method it is using to do
> > the redirect. Is it a 302 response? Is it a meta
> > refresh tag in the html
> > <head>? Is it javascript?  
> > 
> > You can't write code to handle this until you know
> > exactly what you should
> > be looking for in the response. Once you have
> > determined this, you can write
> > code to handle it.
> > 
> > Are you doing a POST or a GET on the failing
> > request? LWP will not by
> > default process a redirect of a POST request. Many
> > web sites use this
> > technique, often when going from a login page to the
> > next page inside the
> > secured area of the application. This can be fixed
> > by overriding a method
> > within UserAgent. I think it is called
> > request_redirectable or something
> > close to that. Search the mailing list archives for
> > code that does this.
> > 
> > If the page is using a meta refresh or some
> > javascript, you can write a
> > regular expression to search for this in the
> > response content & extract the
> > URL to be redirected to. Fairly simple for a meta
> > refresh, maybe not so
> > simple for more complex javascript.
> > 
> > 
> > 
> > ---
> > David Carter
> > [EMAIL PROTECTED]
> > 
> > 
> > > >-----Original Message-----
> > > >From: Ben [mailto:[EMAIL PROTECTED]
> > > >Sent: Tuesday, November 04, 2003 7:51 PM
> > > >To: [EMAIL PROTECTED]; [EMAIL PROTECTED];
> > [EMAIL PROTECTED]
> > > >Subject: RE: Perl Redirect doesn't work
> > > >
> > > >Qing, Keary, and Friends,
> > > >
> > > >I am having the same problem you're having. 
> > Please
> > > >help.
> > > >
> > > >I have a script that gets data from various
> > sites.
> > > >One of the sites suddenly uses a redirect in the
> > url.
> > > >
> > > >I used to go to www.site.com/123.html.  Now
> > > >www.site.com/123.html redirects to
> > > >xyz.site.com/231.html.  The data fields and
> > everything
> > > >else is the same, except I am not able to get any
> > > >information.
> > > >
> > > >Keary mentioned:
> > > >"LWP will only redirect on an HTTP redirect (e.g.
> > > >status 302). Web sites may use other methods to
> > > >simulate a redirect, such as JavaScript or META
> > > >refresh, which LWP doesn't parse and handle."
> > > >
> > > >If this is the case, how can I get around a
> > Javascript
> > > >or META refresh?
> > > >
> > > >Any suggestions or information would be
> > appreciated.
> > > >I am fairly new to Perl. Thank you.
> > > >
> > > >
> > > >Ben
> > > >
> > > >
> > > >-----
> > > >
> > > >
> > > >Hi, All
> > > >     I have one question about the redirection.
> > The
> > > >following is my code:
> > > >
> > > >     use LWP::UserAgent;
> > > >     my $ua=LWP::UserAgent->new;
> > > >     my $req=HTTP::Request->new
> > >
> >
>
>(GET=>'http://www.somedomainhere.com/redirect/mdm2.htm');
> > > >  # this URL should be
> > > >directed to another one
> > > >
> > > >     my $res=$ua->request($req);
> > > >     my $string = $res->content;
> > > >
> > > >     But when I run the program, it didn't
> > > >automatically redirect to another web
> > > >URL. It just show me "The web page has been
> > > >removed.... blah blah". According to
> > > >LWP document, the request "GET" should be in the
> > > >redirect list. Why it doesn't
> > > >work? Could someone help me for this? Thanks!
> > > >
> > > >
> > > >    Qing
> > > >
> > >
> >
>
>http://archive.develooper.com/[EMAIL PROTECTED]/msg04787.html?x
> > 
> > 
> > 

Reply via email to