First, you need to determine exactly how the redirect is being done. Look at the response you are receiving, and determine what method it is using to do the redirect. Is it a 302 response? Is it a meta refresh tag in the html <head>? Is it javascript?
You can't write code to handle this until you know exactly what you should be looking for in the response. Once you have determined this, you can write code to handle it. Are you doing a POST or a GET on the failing request? LWP will not by default process a redirect of a POST request. Many web sites use this technique, often when going from a login page to the next page inside the secured area of the application. This can be fixed by overriding a method within UserAgent. I think it is called request_redirectable or something close to that. Search the mailing list archives for code that does this. If the page is using a meta refresh or some javascript, you can write a regular expression to search for this in the response content & extract the URL to be redirected to. Fairly simple for a meta refresh, maybe not so simple for more complex javascript. --- David Carter [EMAIL PROTECTED] > >-----Original Message----- > >From: Ben [mailto:[EMAIL PROTECTED] > >Sent: Tuesday, November 04, 2003 7:51 PM > >To: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] > >Subject: RE: Perl Redirect doesn't work > > > >Qing, Keary, and Friends, > > > >I am having the same problem you're having. Please > >help. > > > >I have a script that gets data from various sites. > >One of the sites suddenly uses a redirect in the url. > > > >I used to go to www.site.com/123.html. Now > >www.site.com/123.html redirects to > >xyz.site.com/231.html. The data fields and everything > >else is the same, except I am not able to get any > >information. > > > >Keary mentioned: > >"LWP will only redirect on an HTTP redirect (e.g. > >status 302). Web sites may use other methods to > >simulate a redirect, such as JavaScript or META > >refresh, which LWP doesn't parse and handle." > > > >If this is the case, how can I get around a Javascript > >or META refresh? > > > >Any suggestions or information would be appreciated. > >I am fairly new to Perl. Thank you. > > > > > >Ben > > > > > >----- > > > > > >Hi, All > > I have one question about the redirection. The > >following is my code: > > > > use LWP::UserAgent; > > my $ua=LWP::UserAgent->new; > > my $req=HTTP::Request->new > >(GET=>'http://www.somedomainhere.com/redirect/mdm2.htm'); > > # this URL should be > >directed to another one > > > > my $res=$ua->request($req); > > my $string = $res->content; > > > > But when I run the program, it didn't > >automatically redirect to another web > >URL. It just show me "The web page has been > >removed.... blah blah". According to > >LWP document, the request "GET" should be in the > >redirect list. Why it doesn't > >work? Could someone help me for this? Thanks! > > > > > > Qing > > > >http://archive.develooper.com/[EMAIL PROTECTED]/msg04787.html?x
