Re: Sharing memory between children
On Mon, 2003-06-16 at 07:12, Clinton Gormley wrote: I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup) I have gone to great lengths to (1) Preload modules in my startup file (2) Load shared config data during server startup so that that info can also be shared Is it possible that the modules you're using do a lot of work after the compile phase? For example, if they use autoloading or pull in a lot of other modules conditionally later on with require statements that will result in things being compiled after startup and thus not shared. You amount of shared memory seems very low to me, so I think there's something that these modules are doing that is working against sharing. - Perrin
Sharing memory between children
I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup) I have gone to great lengths to (1) Preload modules in my startup file (2) Load shared config data during server startup so that that info can also be shared The only module I load during child init is a wrapper to handle database connections so that database handles don't get shared between children. I've read the memory optimization bit on the mod_perl web site and am complying with all of those suggestions. My next step is to remove all of the modules and start adding them in again to see which has such poor memory usage. What numbers should I be expecting to achieve on a reasonably large web site (http://ww.traveljury.com - still a test site at the moment) and are there any gotchas that you know about that aren't mentioned on the mod_perl site? thanks Clint
Re: Sharing memory between children
On Mon, 2003-06-16 at 12:12, Clinton Gormley wrote:
I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup)
I have gone to great lengths to
(1) Preload modules in my startup file
(2) Load shared config data during server startup so that that info can also be shared
i've pulled everything out and started adding it back in, and it looks like nothing much gets shared other than the perl interpreter.
When I load nothing, the size of my processes starts at 4204 (2504 shared) . When I load everything, the size rises to 17144 (3920 shared) - not much is being shared...
Am I being really dumb here? What am I missing?
thanks
Clint
My Apache compile options :
_
./configure --prefix=/usr/local/apache-1.3.27_perl_ssl_mm \
--activate-module=src/modules/perl/libperl.a \
--disable-module=userdir \
--disable-module=asis \
--disable-module=include \
--enable-shared=info \
--enable-shared=status \
--enable-module=so \
--enable-shared=env \
--enable-shared=setenvif \
--enable-shared=negotiation \
--enable-shared=autoindex \
--enable-shared=access \
--enable-shared=auth \
--enable-shared=cgi \
--enable-shared=actions \
--disable-module=imap \
--enable-module=ssl \
--enable-shared=ssl \
--disable-rule=SSL_COMPAT
_
mod_perl compile options
_
perl Makefile.PL \
APACHE_SRC=../apache_1.3.27/src \
DO_HTTPD=1 \
USE_APACI=1 \
PREP_HTTPD=1 \
EVERYTHING=1
_
My startup file (which is required in the apache config):
_
use strict;
use warnings FATAL = 'all', NONFATAL = 'redefine';
use Apache();
use Apache::DBI();
use lib Apache-server_root_relative ('projects/traveljury/modules');
$Apache::VMonitor::PROC_REGEX = join |, qw(httpd mysql);
## Set DB connection credentials
use TravelJury::Startup::InitDBI();
## Load constants before using Apache::DBI so that $dbh from parent process doesn't get shared
use TravelJury::Startup::Constants();
## Any initialisation which needs to occure during server startup
use TravelJury::Startup::ServerInit();
use TravelJury::Gateway();
Apache-push_handlers(PerlChildInitHandler ='TravelJury::Startup::ChildInit');
_
Re: Sharing memory between children
Hi there, On 16 Jun 2003, Clinton Gormley wrote: I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup) I see about the same on my own server when processes get bloated, but I don't let the processes live to serve large numbers of requests so eventually they're killed off and replaced with new, smaller children. Generally about 25% of the processes are big ones, the rest staying around their initial size, but that's just the way things happen on this server, and it's not especially busy - it runs about half a dozen virtual hosts. Just in case, I put in a lot more RAM than I thought I needed. Of course with a completely different set of (Perl) software and operating conditions, there's no reason why your experience should be anything like this. When I load nothing, the size of my processes starts at 4204 (2504 Sounds a little on the high side, but not outrageous. You're putting quite a lot of stuff into your httpd, do you really need it all? Am I being really dumb here? I don't think so. If it's a problem for your machines you might want to consider a light proxy front end + mod_perl backend. It's easy to set up. A lot depends on the amount and type of traffic that you're going to see. If everything is dynamic HTML you might not gain much, but if there's a lot of static content you might gain a lot. Every site is different - even if they're on the same machine. 73, Ged.
Re: Sharing memory between children
On Mon, 2003-06-16 at 13:03, Ged Haywood wrote: I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup) I see about the same on my own server when processes get bloated, but I don't let the processes live to serve large numbers of requests so eventually they're killed off and replaced with new, smaller children. But this is happening right at startup... before any bloating. Sounds a little on the high side, but not outrageous. You're putting quite a lot of stuff into your httpd, do you really need it all? Sorry - that includes the perl processor. I don't think so. If it's a problem for your machines you might want to consider a light proxy front end + mod_perl backend. I will be using a lightweight frontend, but clearly still want to optimise my mod_perl processes. But I just don't get why I'm seeing the numbers I'm seeing. I preload everything, and yet the shared memory rises all of 1Mb, while the unshared memory rises 12Mb, before the first request has been served! Surely, at this stage, pretty much everything should stil be shared? yours bemusedly Clint
Re: Sharing memory between children
Hi There, On 16 Jun 2003, Clinton Gormley wrote: On Mon, 2003-06-16 at 13:03, Ged Haywood wrote: I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup) But this is happening right at startup... before any bloating. Maybe there are lots of variables in your code that are different for each different process. Do you use the process ID for example? I wouldn't get too excited about it until you see how your servers behave in actual use. I shouldn't be a bit surprised if you find that suddenly there are lots of things that seem to be more important... 73, Ged.
Re: Sharing memory between children
Clinton Gormley wrote: On Mon, 2003-06-16 at 13:03, Ged Haywood wrote: / I had a look at the memory usage of my apache/mod_perl 1 processes, and was alarmed to find that only 3Mb of 25Mb processes was being shared (and that's straight after startup) I see about the same on my own server when processes get bloated, but I don't let the processes live to serve large numbers of requests so eventually they're killed off and replaced with new, smaller children./ But this is happening right at startup... before any bloating. Take all the modules out, and start adding them one by one and see which one causes the bloat, then may be try to split it in chunks and again look at what parts create the bloat. B::TerseSize should be of a great help, it's called from the Apache::Status module, and shows you the exact memory size used by each subroutine, line of code, file, etc. Chapters 9 and 13 in the Practical mod_perl book should give you a good idea on how to use them. There is also some info here: http://perl.apache.org/docs/1.0/guide/performance.html#Measuring_the_Memory_Usage_of_Subroutines __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
