Grabbing a process ID

[Guy Davis]

If I'm using the system command on a linux box is there any way to get the process id 
returned?

I am calling another perl script and it has failed occasionally from errors that I 
have not yet tracked down. What I want to do is grab and store the process id. Then 
check to see if that process id is still running at a later date when the script is 
re-run. 

Another question is there a way to grab the currently running processes on a linux 
system and parse through them in perl? Has anyone done this or do you know where I 
might get the code to do this?

Thanks

Guy Davis


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: Long Time Script Suddenly Failing

[Guy Davis]

Peter,

I'm really pulling my hair out now. I had tried different timeouts (even the default 
of 3 minutes) and still it's not working. Even using your simplified version of the 
script as a test I'm still getting that error so there must be something going on on 
the Linux box I'm using.

I had thought that for some reason our box was no longer recognizing the 
http://inventory page and used the ip address instead. I'm no perl or unix expert and 
am really frustrated but at least it would appear that the problem is definitely on my 
end.

Guy

-Original Message-
From: Peter Scott [mailto:peter@;psdt.com]
Sent: Wednesday, October 23, 2002 11:42 AM
To: [EMAIL PROTECTED]
Subject: Re: Long Time Script Suddenly Failing


In article <[EMAIL PROTECTED]>,
 I wrote:
>I predict that if you insert
>
>   BEGIN { $|=1; print "Content-type: text/plain\n\n" }
>
>after the #! line of that script, you will see something illuminating when
>you next visit it through a browser.

Boy, do I have egg on my face.  I didn't pay attention to the question
and just saw the 500 server error message - thought it was a CGI program,
didn't see that it was browser masquerading.  My apologies to Guy.

I took the script, ripped out the database part, and ended up with what's
below.  It doesn't have the problem described; it fetches a page just fine.
All I can think is that since you set a 15-second timeout that you were
having network problems that day that made it a bit slow to connect to that
site.  Try this again, and if you get the same problem, try accessing the
URL that your program wants to fetch from a browser or with the GET program
that comes with LWP.

The only problem I found was that the script makes a new URI::URL object
and yet there is no "use URI::URL".  So I don't understand how it compiled;
it should have said

 Can't locate object method "new" via package "URI::URL"
 (perhaps you forgot to load "URI::URL"?) 

#!/usr/bin/perl -w
use strict;

use LWP::UserAgent;
use URI::URL;
my $VERBOSE=1;

@MyAgent::ISA = qw(LWP::UserAgent);
sub MyAgent::redirect_ok {
  my ($self,$request)=@_;
  if ($request->method eq "POST") {
$request->method("GET");
  }
  return 1;
}

gettraffic("foo");

sub gettraffic {
my ($keyword) = @_;
my $traffic=0;

my $url =
"http://inventory.overture.com/d/searchinventory/suggestion/?term="; . $keyword;
my $ua  = new MyAgent;
my $agent   = 'Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)';

## Set some User Agent properties.
$ua->agent($agent);
$ua->timeout(15);

## Convert the URL into a url object (with a couple methods we can use).
my $u =  new URI::URL( $url );

## Fetch the web page.
my $req = HTTP::Request->new(GET=>$u->as_string);
my $res = $ua->request($req);
my $resstring = $res->as_string;

print "URL: $url\n" if ($VERBOSE);
print "$resstring\n" if ($VERBOSE);

# Check the outcome of the response
if ($res->is_success) {
my $content = $res->content;
$content =~ m/.* ([0-9]+)<\/b>.*/s;
my $traffic = $1;
if (!$traffic) {
$traffic = 0;
}

} 
}


-- 
Peter Scott
http://www.perldebugged.com

-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Long Time Script Suddenly Failing

[Guy Davis]

Hi all,

I have been requested to repost this message to the begginers group to be worked on 
there.

Hopefully someone can give me a clue about a problem I'm having. I'm using the script 
below to try to retrieve the web page 
"http://inventory.overture.com/d/searchinventory/suggestion/?term="; with a term 
attached at the end of it. This script has been working for months and is all of a 
sudden returning: "500 (Internal Server Error) Can't connect to 
inventory.overture.com:80 (Timeout)"

NOTE: *** This script is not run from a browser but is called from a PHP page and run 
on a Linux box in the background. I've received the above 500 error while trying to 
run it directly from the command line. ***

I can connect to other sites without a problem such as yahoo.com.

Does anyone know why something like this might suddenly be taking place after such a 
long time of working flawlessly? Could it be something has been changed at the web 
site I'm trying to get?

Thanks so much in advance.

Guy Davis





#!/usr/bin/perl 

use strict;
use DBI;
use LWP::UserAgent;

# test to make sure they entered one or two arguments
if($#ARGV != 0)
{
  print "\nUsage: getraffic.pl research_run_id\n\n";
  exit();
}

# We need to subclass LWP::UserAgent in order to allow redirects on POSTS
# and fix some other problems. In other words, to make it behave like a
# 'real' web browser
@MyAgent::ISA = qw(LWP::UserAgent);
sub MyAgent::redirect_ok {
my ($self,$request)=@_;

# *
# IMPORTANT:
#
# POSTs that get redirected __MUST__ change the request
# method to GET
# *
if ($request->method eq "POST") {
$request->method("GET");
}
return 1;
}

# what client do you want to get?  first variable from the command line
my $test_phrase = "";
my $research_id = $ARGV[0];
my $VERBOSE = 1;

my $db = DB_connect();
my ($rp_id, $rp_phrase_name);
my $sql = "";
my $sth;

# Getting started - actively checking for traffic
$sql = "UPDATE research SET research_traffic_updated=1
WHERE research_id=$research_id";
$sth = $db->prepare( $sql );
$sth->execute() || die ( $DBI::errstr );

$sql = "SELECT rp_id, rp_phrase_name FROM researchphrase
WHERE rp_research_id=$research_id AND rp_status<>2 AND rp_clicks IS NULL";

$sth = $db->prepare( $sql );
$sth->execute() || die ( $DBI::errstr );
$sth->bind_columns(undef, \($rp_id, $rp_phrase_name));
while ($sth->fetch()) { # update each phrase
my $kw = $rp_phrase_name;
$kw =~ s/ /+/g;

gettraffic($rp_id, $rp_phrase_name, $kw);
}

# All done - not actively checking for traffic
$sql = "UPDATE research SET research_traffic_updated=0
WHERE research_id=$research_id";
$sth = $db->prepare( $sql );
$sth->execute() || die ( $DBI::errstr );

$sth->finish();
# now disconnect from the database
$db->disconnect();

sub gettraffic {
my ($rp_id, $rp_phrase_name, $keyword) = @_;
my $traffic=0;

my $url = 
"http://inventory.overture.com/d/searchinventory/suggestion/?term="; . $keyword;
my $ua  = new MyAgent;
my $agent   = 'Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)';

## Set some User Agent properties.
$ua->agent($agent);
$ua->timeout(15);

## Convert the URL into a url object (with a couple methods we can use).
my $u =  new URI::URL( $url );

## Fetch the web page.
my $req = HTTP::Request->new(GET=>$u->as_string);
my $res = $ua->request($req);
my $resstring = $res->as_string;

print "URL: $url\n" if ($VERBOSE);
print "$resstring\n" if ($VERBOSE);

# Check the outcome of the response
if ($res->is_success) {
my $content = $res->content;
$content =~ m/.* ([0-9]+)<\/b>.*/s;
my $traffic = $1;
if (!$traffic) {
$traffic = 0;
}

update_db($rp_id, $traffic);
} 
}

sub update_db {
my ($rphraseid, $traffic) = @_;

my $sql = "UPDATE researchphrase SET rp_clicks=$traffic WHERE 
rp_id=$rphraseid";
my $sth = $db->prepare( $sql );
$sth->execute() || die( $DBI::errstr );
$sth->finish();
}

sub DB_connect {
my $table = shift;
my $db = DBI->connect("DBI:mysql:database",  "username", "password", 
{PrintError=>0, RaiseError=>0});
return $db || "DBERROR: Could not connect to database.";
}


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Long Time Script Suddenly Failing

[Guy Davis]

Hi all,

Hopefully someone can give me a clue about a problem I'm having. I'm using the script 
below to try to retrieve the web page 
"http://inventory.overture.com/d/searchinventory/suggestion/?term="; with a term 
attached at the end of it. This script has been working for months and is all of a 
sudden returning: "500 (Internal Server Error) Can't connect to 
inventory.overture.com:80 (Timeout)"

I can connect to other sites without a problem such as yahoo.com.

Does anyone know why something like this might suddenly be taking place after such a 
long time of working flawlessly? Could it be something has been changed at the web 
site I'm trying to get?

Thanks so much in advance.

Guy Davis





#!/usr/bin/perl 

use strict;
use DBI;
use LWP::UserAgent;

# test to make sure they entered one or two arguments
if($#ARGV != 0)
{
  print "\nUsage: getraffic.pl research_run_id\n\n";
  exit();
}

# We need to subclass LWP::UserAgent in order to allow redirects on POSTS
# and fix some other problems. In other words, to make it behave like a
# 'real' web browser
@MyAgent::ISA = qw(LWP::UserAgent);
sub MyAgent::redirect_ok {
my ($self,$request)=@_;

# *
# IMPORTANT:
#
# POSTs that get redirected __MUST__ change the request
# method to GET
# *
if ($request->method eq "POST") {
$request->method("GET");
}
return 1;
}

# what client do you want to get?  first variable from the command line
my $test_phrase = "";
my $research_id = $ARGV[0];
my $VERBOSE = 1;

my $db = DB_connect();
my ($rp_id, $rp_phrase_name);
my $sql = "";
my $sth;

# Getting started - actively checking for traffic
$sql = "UPDATE research SET research_traffic_updated=1
WHERE research_id=$research_id";
$sth = $db->prepare( $sql );
$sth->execute() || die ( $DBI::errstr );

$sql = "SELECT rp_id, rp_phrase_name FROM researchphrase
WHERE rp_research_id=$research_id AND rp_status<>2 AND rp_clicks IS NULL";

$sth = $db->prepare( $sql );
$sth->execute() || die ( $DBI::errstr );
$sth->bind_columns(undef, \($rp_id, $rp_phrase_name));
while ($sth->fetch()) { # update each phrase
my $kw = $rp_phrase_name;
$kw =~ s/ /+/g;

gettraffic($rp_id, $rp_phrase_name, $kw);
}

# All done - not actively checking for traffic
$sql = "UPDATE research SET research_traffic_updated=0
WHERE research_id=$research_id";
$sth = $db->prepare( $sql );
$sth->execute() || die ( $DBI::errstr );

$sth->finish();
# now disconnect from the database
$db->disconnect();

sub gettraffic {
my ($rp_id, $rp_phrase_name, $keyword) = @_;
my $traffic=0;

my $url = 
"http://inventory.overture.com/d/searchinventory/suggestion/?term="; . $keyword;
my $ua  = new MyAgent;
my $agent   = 'Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)';

## Set some User Agent properties.
$ua->agent($agent);
$ua->timeout(15);

## Convert the URL into a url object (with a couple methods we can use).
my $u =  new URI::URL( $url );

## Fetch the web page.
my $req = HTTP::Request->new(GET=>$u->as_string);
my $res = $ua->request($req);
my $resstring = $res->as_string;

print "URL: $url\n" if ($VERBOSE);
print "$resstring\n" if ($VERBOSE);

# Check the outcome of the response
if ($res->is_success) {
my $content = $res->content;
$content =~ m/.* ([0-9]+)<\/b>.*/s;
my $traffic = $1;
if (!$traffic) {
$traffic = 0;
}

update_db($rp_id, $traffic);
} 
}

sub update_db {
my ($rphraseid, $traffic) = @_;

my $sql = "UPDATE researchphrase SET rp_clicks=$traffic WHERE 
rp_id=$rphraseid";
my $sth = $db->prepare( $sql );
$sth->execute() || die( $DBI::errstr );
$sth->finish();
}

sub DB_connect {
my $table = shift;
my $db = DBI->connect("DBI:mysql:database",  "username", "password", 
{PrintError=>0, RaiseError=>0});
return $db || "DBERROR: Could not connect to database.";
}


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Does find_input not work with hidden fields?

[Guy Davis]

Hi all,

I am trying to change a hidden value in a form before the form is posted. Here's a 
slice of code that I'm using. 

my $apclick = $form->find_input('click');
my $apsearch = $form->find_input($search);
my $aptitle = $form->find_input($title);
my $apdesc = $form->find_input($desc);
my $apurl = $form->find_input($url);

$form->value( $apclick, 'true');# this line 
returns error
$form->value( $apsearch, $u_term );
$form->value( $aptitle, $u_title );
$form->value( $apdesc, $u_desc );
$form->value( $apurl, $u_url );
$form->value( $apbid, $u_bid );

The code works for setting the values on the other fields but bombs when trying to set 
the value of the hidden field. The error message returned is:

No such field 'HTML::Form::TextInput=HASH(0x85d5f04)' at ./so.pl line 378

Is there anything special I need to do to change the value of a hidden field?



The reason I'm asking is because I'm populating a form and then calling ->click and 
getting a response of is_succes is true. If I manually try to enter the same data in 
the form when I click on the submit button I get a javascript alert that after 
clicking OK sets the hidden value of the form to 'true' so that when I click the 
submit button again I don't get the alert.

Since I don't think I can change hidden form values how do I get around this annoying 
javascript alert? Any suggestions would be greatly appreciated.

Thanks so much,

Guy Davis


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

How do I get and save an image off of a web page

[Guy Davis]

I want to grab a web page via HTTP:Request and then parse it and grab an image off of 
it and save that image to the hard disk. Has anyone done this and/or can anyone point 
me in the right direction?

Thanks,

Guy Davis


--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Form->Click questions

[Guy Davis]

Hi all,

I'm just learning Perl and have been thrown into the fire so to speak. I have many 
years of programming background but because of my limited experience with Perl and the 
nature of the project I'm working on (wanting results quickly) I'm having some 
questions that are probably pretty obvious to seasoned vets.

I've bastardized someone's code and so far things are working pretty well. In the code 
I open a specific URL using HTML and LWP modules, find a specific form on the page, 
enter a username and password and use ->click to submit the form. The 
$click->is_redirect() function is returning true which is what I want.

But at this point I actually want to grab the redirected page in the same way I had 
grabbed the original page and scan the new page for specific forms that need to be 
filled in. The $click variable appears to have the status of the HTTP form submit. How 
do I get the information off of the redirected page at this point? I get the original 
information using an HTTP::Request. But in this case I'm using the ->click to return a 
page of information.

I apologize if this is making no sense. Since I am using someone else's code I still 
have a lot of fuzziness about what is actually happening. A little nudge in the right 
direction would be helpful and greatly appreciated.

So let me reiterate to make it as clear as possible. 
1) I pass a URL, username, password
2) The URL is opened and scanned for a login form 
3) The form once found is populated with the passed username and password
4) form->click is called which makes the $click->is_redirect() routine equal to true
at this point
5) open the redirected page as in #2 above and do some more form and submit stuff 
(this is where I'm having problems... I don't know how to 'open' this redirected page)

Thanks so much,

Guy Davis
[EMAIL PROTECTED]

The code being used follows:

sub main {
my $loginurl = "https://secure.overture.com/s/dtc/center/";; # LOGIN URL
my ($post_status, $rcode)  = post_message($loginurl, $USER_NAME, 
$USER_PASSWORD, $SEARCH_TERM, $BID_AMOUNT);

### HTTP_error lets you define which HTTP error codes will stop the process.
if ( HTTP_error($rcode) ) {
# return error code and status
print "error ".$post_status."\n";
} else {
# return aok
print "Everything OK!\n";
}
return $rcode;
}

sub post_message {

my $uri = shift;
my $u_name  = shift;
my $u_pass  = shift;
my $u_term  = shift;
my $u_bid   = shift;
my $ua  = shift || new LWP::UserAgent;
my $agent   = shift || 'Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; 
DigExt)';

## Set some User Agent properties.
$ua->agent($agent);
$ua->timeout(30);

## Convert the URL into a url object (with a couple methods we can use).
my $u =  new URI::URL( $uri );

## Fetch the web page.
my $req = HTTP::Request->new(GET=>$u->as_string);
my $res = $ua->request($req);

my @forms;
my $guestbook_html = '';
if ($res->is_success) {
@forms = HTML::Form->parse($res->content, $u->abs);
}
else {
return ('COULD_NOT_FETCH', $res->code);
}

## Get which form object will be used to login.
my $form_num = login_form( @forms );  # Real numbers, starting at 1;
return ('NO_FORMS', $res->code) unless $form_num;

my $form = $forms[$form_num - 1];
my $action = $forms[$form_num - 1]->action;

## Keeps track of the fields we have found.
my %is_field = (
realname=> 0,
url => 0,
comments=> 0,
);

my $fresult;

## Set form values...
($form->value( 'UserName', $u_name), $is_field{UserName} = 1)   if 
$form->find_input( 'UserName', 'text' );
($form->value( '/go2/directraffic/handler/LoginFormHandler.password', 
$u_pass), $is_field{password} = 1)if $form->find_input( 
'/go2/directraffic/handler/LoginFormHandler.password', 'password' );

## If we found the PRIMARY required fields, then let's try to post.
if ($is_field{UserName} && $is_field{password}) {
my $click = $ua->request( $form->click );
if ( $click->is_success() || $click->is_redirect() ) {
### We should have been redirected to another page

my $p = HTML::LinkExtor-&

HELP with accessing HTML page to automate Login using HTML::Form

[Guy Davis]

Hi all,

I'm a complete Perl newbie. I'm ripping apart someones code that accesses a
URL passed in arguments, scans it for forms and specific fields, fills out
those fields and then submits the form.

It works great and I wanted to change it to work with a specific page and
fields to automate a process for the company I work for. Only problem is
that the page is returning a 501 error when accessing the page. The page
happens to be an HTTPS page and I'm wondering if this is happening because
it's a secure part of the site.

I'm fairly confident that this is what's happening but if anyone could
verify that I'd greatly appreciate it. And whether it is or isn't a problem
does anyone have suggestions or clues as to what to do to access the page
through Perl?

Thanks so much,

Guy Davis




-- 
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]