RE: Long Time Script Suddenly Failing

[Guy Davis]

Peter,

I'm really pulling my hair out now. I had tried different timeouts (even the default 
of 3 minutes) and still it's not working. Even using your simplified version of the 
script as a test I'm still getting that error so there must be something going on on 
the Linux box I'm using.

I had thought that for some reason our box was no longer recognizing the 
http://inventory page and used the ip address instead. I'm no perl or unix expert and 
am really frustrated but at least it would appear that the problem is definitely on my 
end.

Guy

-----Original Message-----
From: Peter Scott [mailto:peter@;psdt.com]
Sent: Wednesday, October 23, 2002 11:42 AM
To: [EMAIL PROTECTED]
Subject: Re: Long Time Script Suddenly Failing


In article <[EMAIL PROTECTED]>,
 I wrote:
>I predict that if you insert
>
>       BEGIN { $|=1; print "Content-type: text/plain\n\n" }
>
>after the #! line of that script, you will see something illuminating when
>you next visit it through a browser.

Boy, do I have egg on my face.  I didn't pay attention to the question
and just saw the 500 server error message - thought it was a CGI program,
didn't see that it was browser masquerading.  My apologies to Guy.

I took the script, ripped out the database part, and ended up with what's
below.  It doesn't have the problem described; it fetches a page just fine.
All I can think is that since you set a 15-second timeout that you were
having network problems that day that made it a bit slow to connect to that
site.  Try this again, and if you get the same problem, try accessing the
URL that your program wants to fetch from a browser or with the GET program
that comes with LWP.

The only problem I found was that the script makes a new URI::URL object
and yet there is no "use URI::URL".  So I don't understand how it compiled;
it should have said

 Can't locate object method "new" via package "URI::URL"
 (perhaps you forgot to load "URI::URL"?) 

#!/usr/bin/perl -w
use strict;

use LWP::UserAgent;
use URI::URL;
my $VERBOSE=1;

@MyAgent::ISA = qw(LWP::UserAgent);
sub MyAgent::redirect_ok {
  my ($self,$request)=@_;
  if ($request->method eq "POST") {
    $request->method("GET");
  }
  return 1;
}

gettraffic("foo");

sub gettraffic {
        my ($keyword) = @_;
        my $traffic=0;

        my $url         =
"http://inventory.overture.com/d/searchinventory/suggestion/?term="; . $keyword;
        my $ua          = new MyAgent;
        my $agent       = 'Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)';

        ## Set some User Agent properties.
        $ua->agent($agent);
        $ua->timeout(15);

        ## Convert the URL into a url object (with a couple methods we can use).
        my $u =  new URI::URL( $url );
        
        ## Fetch the web page.
        my $req = HTTP::Request->new(GET=>$u->as_string);
        my $res = $ua->request($req);
        my $resstring = $res->as_string;
        
        print "URL: $url\n" if ($VERBOSE);
        print "$resstring\n" if ($VERBOSE);

        # Check the outcome of the response
        if ($res->is_success) {
                my $content = $res->content;
                $content =~ m/.*&nbsp;([0-9]+)<\/b>.*/s;
                my $traffic = $1;
                if (!$traffic) {
                        $traffic = 0;
                }
                
        } 
}


-- 
Peter Scott
http://www.perldebugged.com

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]