I've scoured deja.com, FAQs, modperl list archives at
forum.swarthmore.edu, 'perldoc mod_perl_traps', experimented ad
nauseum
for 4 days now... this modperl newbie is missing something
important...
Lasting gratitude and a check in the mail for dinner on me to any of
you
who can offer any tips/help which unlock this riddle for me...
Cheers,
Ed Loehr
SYMPTOMS...
-----------
Spurious errors in my error_log with increasingly nasty consequences:
Can't upgrade that kind of scalar at XXX line NN...
Not a CODE reference at XXX line NN...
Modification of a read-only value attempted at XXX line NN...
Attempt to free unreferenced scalar.
Attempt to free unreferenced scalar during global destruction.
Attempt to free unreferenced scalar at XXX line NN...
Once upon a time, the server was fully functional even with these
occasional error messages (which is why I ignored them originally).
Now, they are frequent showstoppers, causing requests to fail
altogether with 500 errors and occasional segfaults...
I'm lumping these together because I suspect they are all related.
In any case, the severest of these at present seems to be the
Can't upgrade that kind of scalar at XXX line NN...
message, which causes the request to fail and seems to foul up that
child for the rest of its life.
FAILED REMEDIES...
------------------
- Turned off PerlFreshRestart
- Got rid of '$| = 1;'
- Got rid of #!/usr/bin/perl -w (!!)
- Check 'use diagnostics' output
- Got rid of string regex optimization flags ( $key =~
m/^xyz/o )
- Replaced use of 'apachectl restart' with
stop-sleep3-startssl;
- use Carp (); local $SIG{__WARN__} = \&Carp::cluck;
- Changed global all instances of global 'my $var = 0' to
'use vars qw($var); $var = 0;'
- Commented out Apache::Registry
(Most of these are just suggestions I found during my hunt...)
CONFIGURATION...
----------------
(detailed config dumps below)
mod_perl 1.21 (*NOT* Apache::StatINC)
mod_ssl 2.4.9
openssl 0.9.4
perl 5.005_03
DBI 1.13 (*NOT* using Apache::DBI)
DBD::Pg 0.92
Apache 1.3.9 (*VERY* lightly loaded)
Linux 2.2.12-20smp (RH 6.1), 1Gb RAM, RAID 5 (*lots* of free
mem)
Dual PII 450 cpus
Using modified TicketMaster scheme from Eagle book
OBSERVATIONS...
---------------
I'm convinced the code referenced by the error msgs (XXX line NN) is
almost at random; typically code that's worked flawlessly before
(sometimes my code, often not). I suspect the line numbers in the
error msg may be screwed up. #line did not clarify things.
If I rearrange my code, I have been able to make the error "move" to
another module (eg., from Exporter.pm to CGI.pm). Smell like a stack
corruption problem? Currently, the first unsuccessful statement is:
# ($r is the usual apache request object)
return ($retval,$msg)
unless $ticket{'ip'} eq
$r->connection->remote_ip;
In -X mode, once the server process hits one of these, it can no
longer
serve any modperl-generated page without a 500 error, occasionally
segfaulting in the process.
This also happens on both production and development server in
slightly
different manifestations (with slightly different httpd.conf files).
Other change factors that may or may not be related: new firewall
rules,
increased number of open file handles (echo 8192 >
/proc/sys/file-max),
increased load, RAM upgrade, numerous modperl app src code changes,
added
more use of Time::HiRes to other modules, new SSL certificate, and
more...
Finally, totally commenting out my incarnation of the TicketMaster
scheme
from the Eagle book (cookie-based passworded sessions) *seems* to
remove
the problem, but it's a moving target so I'm not sure of that yet.
Have
been unable to determine what it might be within TicketMaster that is
causing the problem.
NEXT STEPS...
-------------
Try removing Logger.pm from Apache::Ticket*
Whittle down until minimal set produces error?
Autoload troubles?
Find/try MacEachern's Apache::Leak? Hunting XS errors?
Apache::Vmonitor?
Relying on $_ in foreach loops?
SSL Certificate differences?
Dreaded Last Step: setup debugger and chase ...
# /usr/local/apache_ssl/bin/httpd -l
Compiled-in modules:
http_core.c
mod_env.c
mod_log_config.c
mod_mime.c
mod_negotiation.c
mod_status.c
mod_include.c
mod_autoindex.c
mod_dir.c
mod_cgi.c
mod_asis.c
mod_imap.c
mod_actions.c
mod_userdir.c
mod_alias.c
mod_access.c
mod_auth.c
mod_setenvif.c
mod_ssl.c
mod_perl.c
# /usr/local/apache_ssl/bin/httpd -V
Server version: Apache/1.3.9 (Unix)
Server built: Dec 9 1999 11:40:44
Server's Module Magic Number: 19990320:6
Server compiled with....
-D EAPI
-D HAVE_MMAP
-D HAVE_SHMGET
-D USE_SHMGET_SCOREBOARD
-D USE_MMAP_FILES
-D USE_FCNTL_SERIALIZED_ACCEPT
-D HTTPD_ROOT="/usr/local/apache_ssl"
-D SUEXEC_BIN="/usr/local/apache_ssl/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/httpd.scoreboard"
-D DEFAULT_LOCKFILE="logs/httpd.lock"
-D DEFAULT_XFERLOG="logs/access_log"
-D DEFAULT_ERRORLOG="logs/error_log"
-D TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
-D ACCESS_CONFIG_FILE="conf/access.conf"
-D RESOURCE_CONFIG_FILE="conf/srm.conf"
# perl -v
This is perl, version 5.005_03 built for i386-linux
perl.conf:
**********
PerlRequire conf/startup.pl
<Location /login>
SetHandler perl-script
PerlHandler Apache::TicketMaster
PerlSetVar TicketDomain .xyz.com
PerlSetVar TicketSecret /a/local/secretkey.txt
PerlSetVar TicketExpires 1440
</Location>
<Location /protected>
PerlAccessHandler Apache::TicketAccess
PerlSetVar TicketDomain .xyz.com
PerlSetVar TicketSecret /a/local/secretkey.txt
ErrorDocument 403 https://abc.xyz.com/login
SetHandler perl-script
PerlHandler MyHandler
</Location>
startup.pl:
***********
BEGIN {
use Apache ();
use lib Apache->server_root_relative('lib/perl');
};
# commonly used modules
# use Apache::Registry ();
use Apache::Constants ();
use CGI qw(-compile :all);
1;
