Re: Error Handling/Reporting
Yeah, CGI::Carp will do exactly what you want. I trap all system errors
(DBI, die, etc) using Carp and format the results into a nice webpage for
the user and further, I email myself the error along with the environment
variables in use at the time as well as a stack dump of where it occured.
This has been a life saver for me. I put this in a module that is included
in all perl scripts.
Here's my code snippet that does all the work, it might give you some ideas
.
use CGI::Carp qw(fatalsToBrowser set_message carpout) ;
$SIG{'__DIE__'} = \handle_errors; # This just makes sure we catch
errors and send to handle_errors
# BEGIN ensure this always gets executed first
BEGIN {
sub handle_errors {
$| = 1 ; # Flush all output
my $msg = shift;
my $docroot = $ENV{SITE_ROOT}/html ;
# Default is the default error template
my $file = blank_app_error.html ;
print Can't open error file *$ENV{SITE_ROOT}/html/$file* :: $msg
if !-e $ENV{SITE_ROOT}/html/$file ;
set_message(Error in error handling: $msg) ;
my $text = qq{
System Error: $msg pThis error has already been forwarded
to our support department.p
DBname: $db DBhost: $dbhost Site: $ENV{SITE123}
} ;
my $jstext = $text ;
$jstext =~ s/p/\\n\\n/gsi ;# Convert paragraphs to newlines
$jstext =~ s/(')|()/\\$1/gsi ; # convert embedded quotes with
escaped versions
$jstext =~ s/\015\012/\\n/gsi ; # convert the crlf, otherwise
javascript will fail
$jstext =~ s/\015|\012/\\n/gsi ; # convert the cr or the lf,
otherwise javascript will fail
$jstext =~ s/CODE//gsi ; # remove the CODE tag
# let's make this an alert which ensures its always coming up, even
inside a hidden iframe
$script = scriptalert('$jstext'); /script ;
$text = $script . $text ;
print Content-type: text/html\n\n ;
# Readtemplate is my local routine to read a file and do some custom
parsing on it
my $form = readtemplate($ENV{SITE_ROOT}/html/$file) ;
$form =~ s/ERROR_MSG/$text/ ;
# display message to user
print pre$form ;
## Now we pick up additional information to mail to ourselves to derive
the context
my $stack = CGI::Carp::_longmess() ;# Get the full stack of what
happened
$stack =~ s/eval '.+?PerlRun\.pm line \d+\n//gsi ; # Remove any
evals since we are in PerlRun environment
$text .= \n\npreCall Stack is:\n\n$stack\n\n if $stack ;
# Send/queue an email for this error to the webmaster
# Get context of calling routine
$msg .= \n\n$stack\n\n if $stack ;
if (defined %ENV) {
use Data::Dumper ;
$msg .= \nEnvironment:\n . Dumper(%ENV) ;
}
# My own local interface to sendmail, but essentially you mail yourself
here
email_queue('webmaster','[EMAIL PROTECTED]',$msg,Website Error on
$ENV{SITENAME},$msg) ;
exit 1 ;
}
set_message(\handle_errors) ;
}
- Original Message -
From: Martin Moss [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Tuesday, June 17, 2003 7:23 PM
Subject: Error Handling/Reporting
All,
I've been hunting the web for mechanisms whereby I can trap errors from my
mod perl objects and report them back to a user nicely, log them AND
possibly send reports back to me.
Before I started hunting I thought it would be a simple process to use the
eval mechanism to run my scripts and then report errors back to my
browser.
(replacing the nasty error500 document)
In my travels I found CGI::Carp and Apache::ErrorReport
Before I try to pull them both apart and rebuild something which fits my
requirements, has anybody else done anything along these lines?
Does anybody have any other Recomendations for me to look at?
Kind regards
Marty
Re: Apache modules and mod_perl threads in same process?
Yes, should be the same code. You could, of course, do wierd things in your perl code to pick up different libraries, but I can't imagine you would do that purposefully. dale - Original Message - From: Marc M. Adkins [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Saturday, June 07, 2003 1:21 AM Subject: Apache modules and mod_perl threads in same process? For Apache 2.0.46 / mod_perl 1.99.10-dev: If I have an Apache module configured in httpd.conf, will the module code be executing in the same process as the mod_perl code? If there are multiple Apache processes (on W2K I always see 2) will the module code exist in all processes? mma
Re: Perl Run and Load Average
Assuming your CGI scripts aren't doing strange things, liking hanging around after the session has closed and doing clean-up work, you might check your httpd.conf settings on the number of threads and requests per threads. Its possible that your httpd threads are short-lived and restarting more often which now requires a lot more work to start since Perl is being loaded each time - just an idea. MaxRequestsPerChild 1000# if this were too low, say 10 or 50, you would probably create more load # If these were out of wack somehow, it could possibly create some unusual load conditions MaxSpareServers 20 MinSpareServers 5 I would also run top -d 1 while your webserver is running without mod_perl to see what scripts are causing the load. Do you have a database running on the same system? Do you know if it might be doing more work? Stas already mentioned the pre-loading of modules which will help, but not much if you have a misconfigured httpd.conf as mentioned above. dale - Original Message - From: Batara Kesuma [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, June 05, 2003 5:23 AM Subject: Perl Run and Load Average Hi, I have changed all my CGI scripts to run under Perl Run, and now I notice that the load average of my server (it is a dual CPUs) is very high. It stays around 2.5 all the time. Before, when I was running plain CGI, it was around 0.6 - 1.2. I checked the log file and I found that I only have around 10% more pageviews this week, so the load average shouldn't be that high. Is this normal? Any advice or comment? Thank you. --bk
Re: mod_perl caching form data?
This appears to be the classic global variable/uninitialized variable issue with your script. Mod_perl will load that script once and never reload it again unless you tell it too, even when different users access that script. If you have written a CGI script that doesn't lead itself to a ready rewrite to run under mod_perl, try running it under the PerlRun mode of mod_perl, it works great for this situation. Go here to read up on the issue: http://perl.apache.org/docs/1.0/guide/porting.html#Sometimes_it_Works__Sometimes_it_Doesn_t dale - Original Message - From: David Ressman [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, May 29, 2003 1:19 PM Subject: mod_perl caching form data? Hi all, I'm having a problem with mod_perl 1.2.7 that's baffling me completely, and I've been searching and reading for days, but I still can't figure it out. My apologies if this comes up frequently. I did try rather lengthy searches through the mailing list archives. Right now, I'm using mod_perl 1.2.7 compiled into an apache 1.3.27 server running on a Solaris 9 (semi-current patches, though I can't imagine that that's relevant) server. I've written some fairly straight-forward mod_perl scripts (using CGI.pm). They take form data from the user, process it, and stick it in a database (through DBI). So far, so good. Everything works pretty well... Except that something's caching previously entered form data and displaying it back to me as the default values in those same forms. As an example, this form has a text field that asks for IP addresses, and the text input will occasionally be filled out with the IP address of a system that you had entered a few minutes ago. Naturally, I suspected that my browser was the guilty party, even though I had specified '-1d' as the expiration time in the CGI header() command. It turns out that this is not the case. The forms will occasionally be pre-filled out with IP addresses that other people have given! I even went so far as to set up a network sniffer to verify that the server was indeed setting (in the HTML it sent to the client) the value parameter of the text fields to an IP address that another user had previously entered. Needless to say, my script is *not* setting the default or value parameters for these text fields. As an uneducated guess, I'd say that each httpd child-process is automatically filling out forms with data that it itself has previously received, but that's only a guess, and it still doesn't get me any closer to figuring out why it's happening. Can anyone offer any assistance or point me somewhere that I could find some documentation on what's happening? I'm completely baffled. Thanks! David
Re: Large Data Set In Mod_Perl
I've dealt with fairly large sets, but not as static as yours. If your only
keys for searching are planet and date, then a perl lookup with a hash will
be faster overall since a DB lookup involves connecting to the database,
doing the standard prepare/execute/fetch which could be as costly (for a
single lookup) as the lookup itself. The actual lookup of the record in the
database is probably as fast or faster than Perl (especially after the
initial lookup that primes the caches) if you have indexed the columns on
the table properly.
If you are planning to do lots of lookups on this dataset, preloading the
dataset in a perl hash would definitely be the better approach. If you are
doing only a few lookups over a given period, it may or may not be worth it
and taking up lots of memory for no reason and sticking with the db lookup
would probably be best.
For the perl hash, I would key the hash on the combo of planet and date,
something like:
my %Planets = (
jupiter= {
1900-01-01= ( 5h 39m 18s, +22o
4.0', 28.922, -15,128, -164.799, set),
1900-01-02= ( 5h 39m 18s, +22o
4.0', 28.922, -15,128, -164.799, set),
},
neptune= {
1900-01-01= ( 5h 39m 18s, +22o
4.0', 28.922, -15,128, -164.799, set),
1900-01-02= ( 5h 39m 18s, +22o
4.0', 28.922, -15,128, -164.799, set),
},
) ;
You could also just combine the planet and date as the string for the hash
key like jupiter1900-01-01 but not real sure if this buys you any
performance - it might even be slightly slower since its working on a much
larger single hash rather than a two dimensional hash - might be interesting
to benchmark it on your size dataset to see what really happens. As to
using DB_file, it would probably be somewhere between the Perl hash approach
and using the standard SQL database interface.
dale
- Original Message -
From: simran [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Wednesday, May 28, 2003 9:29 PM
Subject: Large Data Set In Mod_Perl
Hi All,
For one of the websites i have developed (/am developing), i have a
dataset that i must refer to for some of the dynamic pages.
The data is planetary data that is pretty much in spreadsheet format,
aka, i have just under 800,000 rows of data. I don't do any copmlex
searches or functions on the data. I simply need to look at certain
columns at certain times.
sample data set:
planet |date| right_ascension | declination | distance |
altitude | azimuth | visibility
-++-+-+--+
--+--+
jupiter | 1900-01-01 | 15h 57m 7s | -19° 37.2' |6.108 |
10.199 | 39.263 | up
mars| 1900-01-01 | 19h 2m 20s | -23° 36.7' |2.401 |
14.764 |-4.65 | up
mercury | 1900-01-01 | 17h 15m 16s | -21° 59.7' |1.151 |
14.041 | 20.846 | up
moon| 1900-01-01 | 18h 41m 17s | -21° 21.8' | 58.2 |
17.136 |0.343 | transit
neptune | 1900-01-01 | 5h 39m 18s | +22° 4.0' | 28.922
-15.128 | -164.799 | set
I need to be able to say:
* Lookup the _distance_ for the planet _mercury_ on the date _1900-01-01_
Currently i do this using a postgres database, however, my question is,
is there a quicker way to do this in mod_perl - would a DB_File or some
other structure be better?
I would be interested in knowing if others have dealt with large data
sets as above and what solutions they have used.
A DB is quick, but is there something one can use in mod_perl that would
be quicker? perhaps something such as copying the whole 800,000 rows to
memory (as a hash?) on apache startup?
simran.
Boatload of warning messages
Hey, I have combed the various docs and haven't yet found the silver bullet to turn off all the warnings I am getting from mod_perl in my error_log that look something like this: Constant subroutine Apache::ROOTusa_2eusahire_2ecom::cgi_2dbin::portal::gojobs::gojobs_2ecgi::RC_CONTINUE redefined at /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi/Apache/PerlRun.pm line 361. I have tried: PerlWarn Off in the httpd.conf file no warnings ; in the startup.pl no strict ; in the startup.pl setting PERL5OPT to "X" commenting out use strict in PerlRun.pm and this stuff just keeps coming. What's the secret to turning these off? If there isn't anyway, I will have to pipe the logfiles through grep -v and filter them out, a small pain. thanks dale
RE: mod_perl.c Not Compatible with Apache
Trying to narrow down a problem I am having with mod_perl::PerlRun on a
stock RedHat 8.0 system, I have the following that simply doesn't work
reliably under Apache 2.0.40 and mod_perl-1.99_05-3. The script runs fine
for the first few refreshes and then I get the ominous error message:
Error message:
Not a CODE reference at /home/usa/cgi-bin/env.cgi line 7.
After a few retries and then nothing seems to fix it until I do a cold
restart on Apache.
env.cgi is:
#!/usr/bin/perl
print Content-type: text/html\n\n;
require /home/usa/cgi-bin/authlib_test.pl ;
my $status = testsub() ;
foreach (keys %ENV){
print $_ - $ENV{$_}br\n;
}
and the file authlib_test.pl contains:
sub testsub {
return 1 ;
}
1;
If I move the code for authlib_test.pl into env.cgi, it works all the time,
moving it into that seperate file that is required seems to break it. I am
trying to reproduce a code snippet that causes my production mod_perl
programs to produce a zero byte response page (no data returned, but returns
with a status 200). But this is far as I get.
The config is:
LoadModule perl_module modules/mod_perl.so
PerlModule Apache2
Alias /perl /home/usa/cgi-bin
Directory /home/usa/cgi-bin
SetHandler perl-script
PerlHandler ModPerl::PerlRun
PerlOptions +ParseHeaders
PerlSendHeader On
Options ExecCGI FollowSymLinks Includes
/Directory
I also tried this on the CVS code for apache and mod_perl and get the same
results, so I dropped back to the standard RPM versions with RedHat.
Any ideas?
thanks
dale
RE: mod_perl make failed: cannot find -lapr
I want to raise my hand on this one too. I went and downloaded, compiled and built and received the exact same problem on RedHat 8.0 with the same versin of perl and mod_perl (no surprise). But I just don't have the time or expertise to track down the exact reason by libapr.so isn't around or being built. But I did want to add to the issue that its more than just you :-) dale -Original Message- From: Steve Davis [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 31, 2002 2:53 PM To: [EMAIL PROTECTED] Subject: mod_perl make failed: cannot find -lapr Your help will be very much appreciated to resolve the following issue. When attempting to make mod_perl.so, the make script 'almost' makes a touchdown but fails before getting to the finishing line. Now it is time get some help from a coach. Hopefully, with a little help, a touchdown will soon follow. Below, I present was appears to be the problem, but someone else will have to direct me to the next set of steps. I've spend an extensive amount of time trying to resolve this myself and it needs another set of eyes. The news groups were reviewed (repeatedly) for assistance, on-line docs, and what available books I could find. The environment is RH 8.0, Apache 2.0.43, Perl 5.8.0, mod_perl 1.99_07. As best as I can tell, Apache, Perl, and mod_perl have been compiled using the recommended configuration options listed in either Redhat's instructions or mod_perl's on-line docs. Get care has been taken to try to dot every one and cross every 't' to my ability. This problem 'may' have to do with an issue of a change of naming conventions which were adapted by the apr apache group. Confer with Stas Bekman's post on Nov 26, 2002 with a title of Problems compiling mod_perl 1.99_07 in RH 8.0. He provides a cvs patch; but, unfortunately, I'm not familiar with using that-as least-as of yet. (Look's like I might have learn this package real soon.) Are they any intentions to update the mod_perl-1.99_07.tar.gz? If my conclusion is correct, then currently, and according to the on-line instructions, the present tar.gz edition is not compatible to the latest edition of apache. (2.0.43). So, 'maybe' the cause of my make failure. It appears as if, only the cvs repository maybe a valid for compiling. If my analysis is correct, and a new edition of the respective non-cvs files be acquired somehow. I confuse my ignorance and lack of familiarity with these packages so I could be off. Thank you. Steve The environment is described below. Apache has been configured with --prefix=/etc/httpd --with-mpm=prefork. Perl has been configured with -des -Dusethreads -Doptimize='-g' -Dusedevel. The perl Makefile.PL was flagged with MP_AP_PREFIX=/etc/httpd MP_INST_APACHE2=1 MP_APXS=/etc/httpd/bin. (I also, attempted to configure the mod_perl makefile with the MP_APXS which generated the same error message.) The details follow below. Here is excerpt from the screen dump from the make output. make[1] : Leaving directory `/usr/src/mod_perl/mod_perl-1.99_07/WrapXS' make[1] : Entering directory `/usr/src/mod_perl/mod_perl-1.99_07/docs/api/mod_perl-2.0' make[1] : Leaving directory `/usr/src/mod_perl/mod_perl-1.99_07/docs/api/mod_perl-2.0' make[1] : Entering directory `/usr/src/mod_perl/mod_perl-1.99_07/xs' make[2] : Entering directory `/usr/src/mod_perl/mod_perl-1.99_07/xs/APR' make[3] : Entering directory `/usr/src/mod_perl/mod_perl-1.99_07/xs/APR/APR' rm -f ../../../blib/arch/Apache2/auto/APR/APR.so LD_RUN_PATH=/usr/lib cc -shared -L/usr/local/lib APR.o -o ../../../blib/arch/Apache2-lapr -laprutil /usr/bin/ld : cannot find -lapr collect2 : ld returned 1 exit status make[3] : *** [../../../blib/arch/Apache2/auto/APR/APR.so] Error 1 make[3] : Leaving directory `/usr/src/mod_perl/mod_perl-1.99_07/xs/APR/APR' make[2] : *** [subdirs] Error 2 make[2] : Leaving directory `/usr/src/mod_perl/mod_perl-1.99_07/xs/APR' make[1] : *** [subdirs] Error 2 make[1]: Leaving directory `/usr/src/mod_perl/mod_perl-1.99_07/xs' m Here is a list of the files within /etc/httpd/lib. -rw-r--r-- 1 root root 6996 Dec 30 21:04 apr.exp -rw-r--r-- 1 root root 3481 Dec 30 21:04 aprutil.exp -rwxr-xr-x 1 root root 21354 Dec 30 21:01 apxs -rw-r--r-- 1 root root 2685082 Dec 30 21:03 libapr-0.a -rw-r--r-- 1 root root 628 Dec 30 21:03 libapr-0.la lrwxrwxrwx 1 root root 17 Dec 30 01:42 libapr-0.so - libapr-0.so.0.9.2 lrwxrwxrwx 1 root root 17 Dec 30 01:42 libapr-0.so.0 - libapr-0.so.0.9.2 -rwxr-xr-x 1 root root 1282063 Dec 30 01:42 libapr-0.so.0.9.2 -rw-r--r-- 1 root root 1425080 Dec 30 21:04 libaprutil-0.a -rw-r--r-- 1 root root 640 Dec 30 21:04 libaprutil-0.la lrwxrwxrwx 1 root root 21 Dec 30 01:42 libaprutil-0.so - libaprutil-0.so.0.9.2 lrwxrwxrwx 1
RE: RedHat 8.0 standard Apache2.0 and mod_perl 2.0::PerlRun doesn't work?
Thanks for the information. I did try removing Apache:DBI, but it still fails. Building everything from source will be time consuming. If I do that, I will probably just drop back to Apache 1.3 since I know it works. If I can, I will attempt to produce a simple CGI script that fails. The ones that I have in place are simply too complex to send off to you. I will try to narrow it down since I want to stick with the latest stuff. I suspect would of the internal handlers is just not staying hooked up. The debugging show it getting to the eval statement for the script, but it appears as though either the script is empty or it never really eval'd, even though a positive return came back. I'll try to get you something to work with... thanks! dale -Original Message- From: Stas Bekman [mailto:[EMAIL PROTECTED]] Sent: Tuesday, December 24, 2002 8:21 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: RedHat 8.0 standard Apache2.0 and mod_perl 2.0::PerlRun doesn't work? Dale Lancaster wrote: I have searched the archives and various websites to find my problem and its associated resolution to no avail. I upgraded my working Apache 1.3 and mod_perl 1.x website using the PerlRun option of modperl to the RedHat 8.0 standard release with Apache/mod_perl 2.0 combo -- bad move it appears. ModPerl::Registry and friends are in beta now. I've ported the 1.0's version and made it a bit more modular via ModPerl::RegistryCooker. It's quite possible that I broke some of the functionalities while porting as you saw in the recent bug fix. The problem is that I couldn't verify whether the porting was 1:1 because there was no exhaustive test suite. For 2.0 we are hoping to have one. First, disable Apache::DBI. Does the problem persist? Second, please download the latest cvs versions of apache and mod_perl and try again. Does the problem persist? If it does, please send a short script that reproduces the problem, and hopefully a self-contained one, so it's not relying on mysql. In short, think of it this way: To solve the problem, I need to reproduce it first. So help me to accomplish that first. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
RedHat 8.0 standard Apache2.0 and mod_perl 2.0::PerlRun doesn't work?
I have
searched the archives and various websites to find my problem and its associated
resolution to no avail.
I
upgraded my working Apache 1.3 and mod_perl 1.x website using the PerlRun option
of modperl to the RedHat 8.0 standard release with Apache/mod_perl 2.0 combo --
bad move it appears.
When I
start mod_perl on my website, the first couple of CGI loads on a given script
work and then I begin to receive blank page responses. Meaning,I
refresh the page and the response is just blank. No errors, no text (other
than a standard header). Looking in the log file all I see
is:
[24/Dec/2002:12:56:10 -0500] "GET /cgi-bin/filter.cgi?Mode=ListApplicants
HTTP/1.1" 200 0 "http://mysite.com/cgi-bin/jobs.cgi?Mode=DisplayResponses" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;
Q312461)"
and the error log
indicates no errors at all. Note the above that I have a 200 status with 0
bytes, when in reality when it does work, it should show several thousand bytes
of transfer. After a few more refreshes, I get output that I expect.
This happens with all of my scripts, not just the same one. If I restart
the server, it always works the first time. Invariably the first and
subsequent refreshes tend to fail (produce blank pages).
I have enabled any
and all types of debugging. I found in RegisteryCooker where I could
enable some debugging and it shows it going through, loading and compiling the
script, running it and then flushing the namespace, but no HTML output
occurs.
I am using the
prefork() MPM and fiddled with various settings to no avail. I have
ensured that caching is disabled, I have played with mod_expire settings, moved
the mod_perl module to be loaded first and all kinds of stuff just to try and
get the behavior to change, all to no avail as well. I also used some
suggestions on how to use the mod_perl::Registry with settings to force it to
restart threads everytime and it doesn't work either. My scripts all use
DBI with mysql. I have noticed that the mysql server is not being hit when
the script produces a blank response. So that tells me that either the
"eval{}" call in RegistryCooker is really not working somehowor the script
is simply not running, but producing no errors anywhere. I don't know why
the eval{} call to run my script would work the first time and fail the second
time, strange.
My perlrun
configuration is:
PerlModule
Apache2PerlModule Apache::DBIPerlRequire
/etc/httpd/conf.d/startup.pl
Directory /home/usa/cgi-bin
SetHandler perl-script PerlHandler
ModPerl::PerlRun PerlSendHeader On
PerlOptions +ParseHeaders Options ExecCGI FollowSymLinks
Includes/Directory
startup.pl
contains:
# This will give
us a stack trace when a perl script dies#$SIG{__WARN__} =
\Carp::cluck;
# Load CGI.pm and
call its compile() method to precompile# (but not to import) its autoloaded
methods.use CGI ();CGI-compile(':all');
use Apache::DBI()
;use DBI() ;
All the scripts run
fine without mod_perl, and in fact ran just fine on the old Apache 1.x system
with mod_perl. I am at the end of my wits on this one and looking for
ideas.
Thanks
dale
