On Mon, 29 May 2000, Vivek Khera wrote:
> >>>>> "SB" == Stas Bekman <[EMAIL PROTECTED]> writes:
>
> SB> A while ago, a few people have mentioned that it's possible to improve the
> SB> way Perl data structures get mapped in memory pages, by exercising the
> SB> code before the child processes have been spawned, i.e. running the code
> SB> and not just pre-compiling it. Did anyone try this at home :) Any
> SB> satisfactionary results on this direction or what it just a crazy idea?
>
> This is extremely important for DBI, since the DBD layer is not loaded
> until needed. Thus, unless you exercise your Connect in DBI, the DBD
> is not loaded until each child is spawned. I'm sure other modules do
> some sort of initialization as well which could break sharing since
> perl code and data pages are the same as far as the OS is concerned.
In fact it's not the connect, but the install_driver that makes the
difference, connect_on_init (I know you said connect, not on_init :) only
gets the connection ready for the first request for each process, it
doesn't preload the driver!!! surprise, surprise... rush and update your
startup files!
Here are the tests I've conducted. Our goal is to find the startup
environment tha will lead to the smallest "Difference" between the shared
and normal memory reports, therefore lesser total memory usage.
Here are the results (exclusively for the mod_perl list ;-) :
The startup.pl file could have:
nothing added 1
install_driver 2
connect_on_init 3
Results (sorted by Diff):
=== First request:
Test: Size Shared Diff
---------------------------------------
2 : 3465216 2621440 843776 (bytes)
2+3 : 3461120 2609152 851968 (bytes)
1 : 3461120 2494464 966656 (bytes)
3 : 3461120 2482176 978944 (bytes)
=== Second request (all the subsequent request showed the same results):
Test: Size Shared Diff
---------------------------------------
2 : 3469312 2609152 860160 (bytes)
2+3 : 3481600 2605056 876544 (bytes)
1 : 3477504 2482176 995328 (bytes)
3 : 3481600 2469888 1011712 (bytes)
Notice that after the second reload the process's memory size grows a
little bit and there are less shared memory pages, but all the subsequent
reloads have shown the same results.
As you can see that the *real* winner is the startup.pl file that has used
install_driver (2). Having only connect_on_init (3) is the worst case in
terms of shared memory usage.
Here is the test enviroment:
### startup.pl:
* Always preloaded:
Gtop() and Apache::DBI()
* install_driver (2):
DBI->install_driver("mysql");
* connect_on_init (3):
Apache::DBI->connect_on_init('DBI:mysql:test::localhost',
"",
"",
{
PrintError => 1, # warn() on errors
RaiseError => 0, # don't die on error
AutoCommit => 1, # commit executes
# immediately
}
)
or DBI->disconnect("Cannot connect to database: $DBI::errstr\n");
### httpd.conf:
MinSpareServers 1
MaxSpareServers 1
StartServers 1
MaxClients 1
MaxRequestsPerChild 100
PerlModule Apache::Registry
### The registry script used in test (unmodified in all 4 test):
preload_dbi.pl
--------------
use strict;
use GTop ();
use DBI ();
my $dbh = DBI->connect("DBI:mysql:test::localhost",
"",
"",
{
PrintError => 1, # warn() on errors
RaiseError => 0, # don't die on error
AutoCommit => 1, # commit executes
# immediately
}
)
or DBI->disconnect("Cannot connect to database: $DBI::errstr\n");
my $r = shift;
$r->send_http_header('text/plain');
my $do_sql = "show tables";
my $sth = $dbh->prepare($do_sql);
$sth->execute();
my @data = ();
while (my @row = $sth->fetchrow_array){
push @data, @row;
}
$dbh->disconnect();
print "Data: @data\n";
my $proc_mem = GTop->new->proc_mem($$);
my $size = $proc_mem->size;
my $share = $proc_mem->share;
my $diff = $size - $share;
printf "%8s %8s %8s\n", qw(Size Shared Diff);
printf "%8d %8d %8d (bytes)\n",$size,$share,$diff;
---------------------
Your comments are very welcome!
Especially about other modules that allow initialization and thus
reducing the memory usage. (not CGI.pm, I know about this one)
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:[EMAIL PROTECTED] http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
