Re: Mod_perl 1.99 spawning processes causing major performanceissue
Thanks for using REPORT!
On Mon, 2003-06-09 at 16:07, George Bagley wrote:
> On Apache 1.3, when I do a ps -ef, I cannot see the cgi script running.
> I assume this is because Apache is NOT spawning a separate process to
> satisfy the request.
>
> On Apache2, there are hundreds of the cgi scripts running and
> performance is roughly half what it was on Apache 1.3
Sounds like your scripts are not actually running under mod_perl. To
test this, you can try printing out the value of $ENV{'MOD_PERL'} in one
of them. If you're running under mod_perl (via ModPerl::Registry) that
variable will be defined.
> # PerlModule ModPerl::Registry
>
> AddHandler cgi-script cgi pl
> # AddHandler perl-script .pl
> # PerlResponseHandler Apache::Registry
> PerlResponseHandler ModPerl::Registry
> PerlOptions +ParseHeaders
> AllowOverride None
> Options +ExecCGI -Includes
> # SetHandler perl-script
> SetHandler cgi-script
>
That last line is the problem. The perl-script one should be
uncommented, not the cgi-script one.
- Perrin
Mod_perl 1.99 spawning processes causing major performance issue
-8<-- Start Bug Report 8<-- 1. Problem Description: Hi I have ugraded from apache1.3 to Apache2 and I am having a performance problem with a cgi script. Hardware Dell 2550, Dual Proc, 2GB RAM, RedHat Linux 9.0 On Apache 1.3, when I do a ps -ef, I cannot see the cgi script running. I assume this is because Apache is NOT spawning a separate process to satisfy the request. On Apache2, there are hundreds of the cgi scripts running and performance is roughly half what it was on Apache 1.3 The cgi script in question makes a connection to a mysql database. I have had to increase the max number of connections allowed by mysql to allow for the large number of spawned perl processes attempting to make a DB connection. Is there a configuration option to stop this happening? I have the following relating to perl in my httpd.conf # LoadModule foo_module modules/mod_foo.so # LoadModule perl_module modules/mod_perl.so # PerlRequire "/usr/local/apache2/2.0.45/conf/startup.pl" PerlModule Apache2 # PerlModule ModPerl::Registry AddHandler cgi-script cgi pl # AddHandler perl-script .pl # PerlResponseHandler Apache::Registry PerlResponseHandler ModPerl::Registry PerlOptions +ParseHeaders AllowOverride None Options +ExecCGI -Includes # SetHandler perl-script SetHandler cgi-script 2. Used Components and their Configuration: *** using lib/Apache/BuildConfig.pm *** Makefile.PL options: MP_AP_PREFIX=> /usr/local/apache MP_COMPAT_1X=> 1 MP_GENERATE_XS => 1 MP_INST_APACHE2 => 1 MP_LIBNAME => mod_perl MP_USE_DSO => 1 MP_USE_STATIC => 1 *** /usr/local/apache/bin/httpd -V Server version: Apache/2.0.45 Server built: Apr 30 2003 17:51:58 Server's Module Magic Number: 20020903:0 Architecture: 32-bit Server compiled with -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_PROC_PTHREAD_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D HTTPD_ROOT="/usr/local/apache2/2.0.45" -D SUEXEC_BIN="/usr/local/apache2/2.0.45/bin/suexec" -D DEFAULT_PIDLOG="logs/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" *** /usr/bin/perl -V Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration: Platform: osname=linux, osvers=2.4.20-8smp, archname=i686-linux uname='linux redeye-lin-01.pajmc.com 2.4.20-8smp #1 smp thu mar 13 17:45:54 est 2003 i686 i686 i386 gnulinux ' config_args='-d -Dprefix=/usr' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-O3', cppflags='-fno-strict-aliasing -I/usr/include/gdbm' ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: USE_LARGE_FILES Built under linux Compiled at Apr 30 2003 17:16:30 %ENV: PERL_LWP_USE_HTTP_10="1" @INC: /usr/lib/perl5/5.8.0/i686-linux /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i686-linux /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl 3. This is the core dump trace: (if you get a core dump): I am not getting an ERROR. Just a major performance issue. This report was generated by t/REPORT on Mon Jun 9 20:59:06 2003 GMT. -8<-- End Bug Report --8<-- Note: Complete the rest of the details and post this bug report to dev perl.apache.org. To subscribe to the list send an empty email to [EMAIL PROTECTED]
Re: Mod_perl spawning processes
On Mon, 2003-06-09 at 15:34, George Bagley wrote: > CONFIG redhat linux 9.0 > apache 2 I'm afraid that's not enough info to guess what you're doing. Please read http://perl.apache.org/docs/2.0/user/help/help.html#Reporting_Problems > I have ugraded from apache1.3 to Apache2 and I am having a performance > problem with a cgi script. Do you mean a script running under ModPerl::Registry? We don't support actual CGI on this list, just mod_perl. - Perrin
Mod_perl spawning processes
Hi CONFIG redhat linux 9.0 apache 2 I have ugraded from apache1.3 to Apache2 and I am having a performance problem with a cgi script. On 1.3, when I do a ps, I cannot see the cgi script running. On 2, there are hundreds of the cgi scripts running. The cgi makes a connection to a mysql database. Is there a configuration option to stop this happening? Thanks George Bagley Senior Systems Analyst Red Eye International Ltd 24-28 Nelson's Row Clapham London SW4 7JT d: 020 7627 9313 t: 020 7627 9300 f: 020 7627 9301 e: [EMAIL PROTECTED] n: www.redeye.com eCRM experts
Re: pre-spawning database connections[newbie]
Sven Geisler wrote: Hi Ben, Do you use Apache::DBI? I mean yes because you're using connect_on_init. Apache::DBI do not really close your DBI connection. You will get the same connection with the same connection parameters, when you call DBI->connect. All connections are cached by Apache::DBI. Yes, you should call DBI->connect. Each sub process will get it's own connections. What Sven said, plus it helps to RTFM before asking the list. Please read: http://perl.apache.org/docs/1.0/guide/databases.html#Apache__DBI___Initiate_a_persistent_database_connection Am Mon, 2003-01-13 um 13.08 schrieb Ben Wrigley: Hi All, I'm a mod_perl newbie and just trying to understand a little more about the startup.pl files and prespawning databases. I am using the connect_on_init routine in the startup.pl which is fine. What I'm not sure is then how to use this most economically in my scripts. It seems that in the scripts you should call the DBI->connect again, but that seems to be bypassing the connection I made at startup or am I misunderstanding completely. Thanks for your help Ben -- __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: pre-spawning database connections[newbie]
Hi Ben, Do you use Apache::DBI? I mean yes because you're using connect_on_init. Apache::DBI do not really close your DBI connection. You will get the same connection with the same connection parameters, when you call DBI->connect. All connections are cached by Apache::DBI. Yes, you should call DBI->connect. Each sub process will get it's own connections. Regards Sven. Am Mon, 2003-01-13 um 13.08 schrieb Ben Wrigley: > Hi All, > > I'm a mod_perl newbie and just trying to understand a little more about the >startup.pl files and prespawning databases. > > I am using the connect_on_init routine in the startup.pl which is fine. > > What I'm not sure is then how to use this most economically in my scripts. > > It seems that in the scripts you should call the DBI->connect again, but that seems >to be bypassing the connection I made at startup or am I misunderstanding completely. > > Thanks for your help > > Ben
pre-spawning database connections[newbie]
Hi All, I'm a mod_perl newbie and just trying to understand a little more about the startup.pl files and prespawning databases. I am using the connect_on_init routine in the startup.pl which is fine. What I'm not sure is then how to use this most economically in my scripts. It seems that in the scripts you should call the DBI->connect again, but that seems to be bypassing the connection I made at startup or am I misunderstanding completely. Thanks for your help Ben
Re: Spawning
On Thu, 12 Oct 2000 [EMAIL PROTECTED] wrote: > Thanks for your reply, > I read through the documentation and IPC-Shareable is exactly what I > need. I am having some problems getting it working. (Maybe that's why you > wished me Good luck). Seems to have to do with the size option. I can tie > simple variables but I get "Munged shared memory segment (size exceeded?)" > when I try to tie a package. Is that not possible or can I fiddle with the > size option? The default however would seem to be adequate (65536). I got the same error constantly when attempting to retrieve a key's value (or testing with exists()) after it had been delete()ed. After various different attempts at getting it working over 3-4 days I eventually gave up. I hadn't exceeded any size constraint. See if IPC::ShareLite does what you need, for the record I ended up using IPC::SharedCache, which does almost all of what I wanted. -- steven
Re: Spawning
--- On 10/10/2000 11:46:14 PM bcburke wrote: --- >You can use Perl's IPC::Shareable to share objects in memory across >processes: >http://theoryx5.uwinnipeg.ca/CPAN/data/IPC-Shareable/IPC/Shareable.html >Good luck, Thanks for your reply, I read through the documentation and IPC-Shareable is exactly what I need. I am having some problems getting it working. (Maybe that's why you wished me Good luck). Seems to have to do with the size option. I can tie simple variables but I get "Munged shared memory segment (size exceeded?)" when I try to tie a package. Is that not possible or can I fiddle with the size option? The default however would seem to be adequate (65536). And again thanks for the idea. Atli.
Re: Spawning
You can use Perl's IPC::Shareable to share objects in memory across processes: http://theoryx5.uwinnipeg.ca/CPAN/data/IPC-Shareable/IPC/Shareable.html Good luck, Brian B. [EMAIL PROTECTED] - Original Message - From: <[EMAIL PROTECTED]> To: Greg Cope <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Wednesday, October 11, 2000 10:11 AM Subject: Re: Spawning > > > >I would always seperate any proccess that are not web request orientated > >outside the request architecture - i.e write your own daemon / crond > >script thats does this exterior processing. > > I totally agree. That's the plan but I need the Apache connection for the > external processing > > >What do you need to access apache internals for ? > > I have an object(package) that can only have one instance running. That > object is currently created by startup.pl. I need to access that instance > from apache modules > AND the external script. > > >Starting something on startup is just a question of adding the required > >lines to the server startup script. > > Yup, I did that and it works fine. Just need to make that connection. I > don't think calling system(...) in startup.pl and setsid() in the script > would help because that would just start up another perl session > disconnected from Apache... am I right? I am unfortunately not too familiar > with the intricacies of the Perl/Apache connection. > Atli. > > >
Re: Spawning
>I would always seperate any proccess that are not web request orientated >outside the request architecture - i.e write your own daemon / crond >script thats does this exterior processing. I totally agree. That's the plan but I need the Apache connection for the external processing >What do you need to access apache internals for ? I have an object(package) that can only have one instance running. That object is currently created by startup.pl. I need to access that instance from apache modules AND the external script. >Starting something on startup is just a question of adding the required >lines to the server startup script. Yup, I did that and it works fine. Just need to make that connection. I don't think calling system(...) in startup.pl and setsid() in the script would help because that would just start up another perl session disconnected from Apache... am I right? I am unfortunately not too familiar with the intricacies of the Perl/Apache connection. Atli.
Re: Spawning
[EMAIL PROTECTED] wrote: > > Hi there, > I am working on a web-oriented game that runs on Apache/modperl and > MySQL. I have one perl process that runs forever and is outside of Apache > that does a lot of tallying up and cleaning. I want the process to be > started by Apache on server startup and give the process access to Apache > much like a normal modperl script. Is this possible? > Atli. I would always seperate any proccess that are not web request orientated outside the request architecture - i.e write your own daemon / crond script thats does this exterior processing. What do you need to access apache internals for ? Starting something on startup is just a question of adding the required lines to the server startup script. > p.s. I was able to achieve something similar by placing the script in my > modules directory and simply calling it from a browser. The script never > returns so the browser hangs but at least the script runs. This is however > rather messy and obviously not a method I want to use. Not highly recomended but what about an exec / system call from the startup.pl ? (as root remeber!). Greg Cope
Spawning
Hi there, I am working on a web-oriented game that runs on Apache/modperl and MySQL. I have one perl process that runs forever and is outside of Apache that does a lot of tallying up and cleaning. I want the process to be started by Apache on server startup and give the process access to Apache much like a normal modperl script. Is this possible? Atli. p.s. I was able to achieve something similar by placing the script in my modules directory and simply calling it from a browser. The script never returns so the browser hangs but at least the script runs. This is however rather messy and obviously not a method I want to use.
Re: Apache::Registry spawning zombie shells?
On Tue, 22 Aug 2000, martin langhoff wrote:
> hi list,
>
> while doing a silly thing (building a set of HTML files with info from
> a DB file), I found that while the apache server was being crawled by
> lwp-rget, a lots of zombie shells were being spawned and killed.
> sh -c /bin/csh -cf 'set nonomatch; glob /table' 2>/de
it looks like one of your html tags is astray, which Perl interprets as
a <> glob, e.g.
% strace -o strace.out -f /usr/bin/perl -e ''
% grep glob strace.out
10627 execve("/bin/csh", ["/bin/csh", "-cf", "set nonomatch; glob /table"], [/* 56
vars */]) = 0
i don't see any in your script that would trigger that, but you might want
to double check where all of your tags are.
Apache::Registry spawning zombie shells?
hi list,
while doing a silly thing (building a set of HTML files with info from
a DB file), I found that while the apache server was being crawled by
lwp-rget, a lots of zombie shells were being spawned and killed.
top was telling me that there were quite a few processes like:
6766 nobody 2 0 00 0 Z 0 0.3 0.0 0:00 sh
(I was looking at top because I was happy thinking how mod_perl was
speeding it all up)
so I checked and rechecked my code, 2 *very* silly cgi scripts hacked
in a hurry, that I renamed from .cgi to .pl so the crawling finished
faster. the scripts are at the bottom, but I don't think you'll find
much there.
needless to say, when run under mod_cgi, no sh is ever spawned, and
when the crawling finished no more spawning took place (3 of the zombie
shells remained, though). So it's definitely something with mod_perl and
Apache::Registry. Maybe the DB_File module has something blame? Don't
really know.
these shells are not attached to any console, and they are sh shells,
while I use bash. queer. If it weren't because the server is
*disconnected* from the 'net I'd think the box was 'rooted'.
I've even got to catch the actuall call to sh from ps:
sh -c /bin/csh -cf 'set nonomatch; glob /table' 2>/de
what can that mean? I'm off to search ...
martin
system specs and scripts : 'Apache/1.3.12 (Unix) mod_perl/1.24'
--
Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration:
Platform:
osname=linux, osvers=2.2.5-22smp, archname=i386-linux
uname='linux porky.devel.redhat.com 2.2.5-22smp #1 smp wed jun 2
09:11:51 ed
t 1999 i686 unknown '
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef useperlio=undef d_sfio=undef
Compiler:
cc='cc', optimize='-O2', gccversion=egcs-2.91.66 19990314/Linux
(egcs-1.1.2
release)
cppflags='-Dbool=char -DHAS_BOOL -I/usr/local/include'
ccflags ='-Dbool=char -DHAS_BOOL -I/usr/local/include'
stdchar='char', d_stdstdio=undef, usevfork=false
intsize=4, longsize=4, ptrsize=4, doublesize=8
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
alignbytes=4, usemymalloc=n, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -ldl -lm -lc -lposix -lcrypt
libc=, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Built under linux
Compiled at Aug 30 1999 23:09:51
@INC:
/usr/lib/perl5/5.00503/i386-linux
/usr/lib/perl5/5.00503
/usr/lib/perl5/site_perl/5.005/i386-linux
/usr/lib/perl5/site_perl/5.005
script letra.pl:
---
#!/usr/bin/perl -w
use strict;
use DB_File;
use CGI;
use URI::Escape;
my $cgi = new CGI;
print $cgi->header();
my $letra = $cgi->param('letra');
my %db;
my $records = 0;
tie (%db, "DB_File", 'voluntarios.db') or die $!;
print '';
print '';
foreach my $key (sort { lc($a) cmp lc($b) } keys %db){
if ($key =~ /^$letra/io){
$records++;
print '
';
print $cgi->a( {href=>'record.pl?record=' . uri_escape($key)} ,
qq{$key});
print '';
}
};
unless ($records){
$letra = uc $letra;
print '
';
print qq{No hay registros con la letra $letra};
print '';
}
print '';
print '';
--
script record.pl
--
#!/usr/bin/perl -w
use strict;
use DB_File;
use CGI;
use URI::Escape;
my $cgi = new CGI;
print $cgi->header();
my $record = $cgi->param('record');
my %db;
tie (%db, "DB_File", 'voluntarios.db') or die $!;
print '
