On Monday 08 August 2005 01:30, Stas Bekman wrote: > Torsten Foertsch wrote: > > Hi, > > > > the attached patch allows Apache2::SizeLimit to use the new > > /proc/PID/smaps instead of /proc/PID/statm. This takes into account > > copy-on-write pages when counting shared memory. > > Torsten++! > > Could you please submit a doc patch for Apache2/SizeLimit.pod > http://svn.apache.org/viewcvs.cgi/perl/modperl/docs/trunk/src/docs/2.0/api/ >Apache2/SizeLimit.pod?rev=159831&view=log explaining the fact that shared > memory calculation is normally wrong under linux kernel 2.6 and how to fix > that situation? (below)
see attachment It corrects some podchecker errors and warnings as well. > > The patch looks if /proc/PID/smaps exists and if Linux::Smaps is > > installed. If not it uses the old /proc/PID/statm. Hence, it is > > compatible with old linuxes. > > > > /proc/PID/smaps exists by now in the mm-series of the linux kernel. I > > hope the patch will take its way into the vanilla kernel in the not so > > distant future. > > [...] > > > The first one uses linux 2.6.13-rc4-mm1 (with a little patch that will go > > into the next mm-version, see > > http://marc.theaimsgroup.com/?l=linux-kernel&m=112335328222533&w=2), the > > second with Suses default kernel (2.6.11.4-20a-default). > > Do you know when the mm-series will become a part of the mainstream branch? MM (Andrew Morton) says: "Yes, I expect we'll merge it up.". Since he is the maintainer of the 2.6 vanilla linux I think it can be expected rather sooner than later. Torsten
--- mod_perl-2.0.1/docs/api/Apache2/SizeLimit.pod~ 2005-04-26 20:58:54.000000000 +0200
+++ mod_perl-2.0.1/docs/api/Apache2/SizeLimit.pod 2005-08-19 18:21:11.726281299 +0200
@@ -2,8 +2,6 @@
Apache2::SizeLimit - Because size does matter.
-
-
=head1 Synopsis
This module allows you to kill off Apache httpd processes if they grow
@@ -61,7 +59,7 @@
This module is highly platform dependent, please read the
L<Caveats|/Caveats> section. It also does not work L<under threaded
-MPMs|/Supported_MPMs>.
+MPMs|/Supported MPMs>.
This module was written in response to questions on the mod_perl
mailing list on how to tell the httpd process to exit if it gets too
@@ -138,6 +136,81 @@
seems to be fast enough on modern systems. If you are worried about
performance, try setting the C<CHECK_EVERY_N_REQUESTS> option.
+Since linux 2.6 F</proc/self/statm> does not report the amount of
+memory shared by the copy-on-write mechanism as shared memory. Hence
+decisions made on the basis of C<MAX_UNSHARED_SIZE> or C<MIN_SHARE_SIZE>
+are inherently wrong.
+
+To correct the situation there is a patch to the linux kernel that adds a
+F</proc/self/smaps> entry for each process. At the time of this writing
+the patch is included in the mm-tree (linux-2.6.13-rc4-mm1) and is expected
+to make it into the vanilla kernel in the near future.
+
+F</proc/self/smaps> reports various sizes for each memory segment of a
+process and allows to count the amount of shared memory correctly.
+
+If C<Apache2::SizeLimit> detects a kernel that supports F</proc/self/smaps>
+and if the C<Linux::Smaps> module is installed it will use them instead of
+F</proc/self/statm>.
+
+NOTE: Reading F</proc/self/smaps> is very expensive compared to
+F</proc/self/statm>. It must look at each page table entry of a process.
+Further, on multiprocessor systems the access is synchronized with
+spinlocks. Hence, you are very encouraged to set the C<CHECK_EVERY_N_REQUESTS>
+option.
+
+The following example shows the effect of copy-on-write:
+
+ <Perl>
+ require Apache2::SizeLimit;
+ package X;
+ use strict;
+ use Apache2::RequestRec ();
+ use Apache2::RequestIO ();
+ use Apache2::Const -compile=>qw(OK);
+
+ my $x="a"x(1024*1024);
+
+ sub handler {
+ my $r=shift;
+ my ($size, $shared)=$Apache2::SizeLimit::HOW_BIG_IS_IT->();
+ $x=~tr/a/b/;
+ my ($size2, $shared2)=$Apache2::SizeLimit::HOW_BIG_IS_IT->();
+ $r->content_type('text/plain');
+ $r->print("1: size=$size shared=$shared\n");
+ $r->print("2: size=$size2 shared=$shared2\n");
+ return Apache2::Const::OK;
+ }
+ </Perl>
+
+ <Location /X>
+ SetHandler modperl
+ PerlResponseHandler X
+ </Location>
+
+The parent apache allocates a megabyte for the string in C<$x>. The
+C<tr>-command then overwrites all "a" with "b" if the handler is
+called with an argument. This write is done in place, thus, the
+process size doesn't change. Only C<$x> is not shared anymore by
+means of copy-on-write between the parent and the child.
+
+If F</proc/self/smaps> is available curl shows:
+
+ [EMAIL PROTECTED]:~/work/mp2> curl http://localhost:8181/X
+ 1: size=13452 shared=7456
+ 2: size=13452 shared=6432
+
+Shared memory has lost 1024 kB. The process' overall size remains unchanged.
+
+Without F</proc/self/smaps> it says:
+
+ [EMAIL PROTECTED]:~/work/mp2> curl http://localhost:8181/X
+ 1: size=13052 shared=3628
+ 2: size=13052 shared=3636
+
+One can see the kernel lies about the shared memory. It simply doesn't count
+copy-on-write pages as shared.
+
=item Solaris 2.6 and above
For Solaris we simply retrieve the size of F</proc/self/as>, which
@@ -148,7 +221,7 @@
the /proc filesystem has changed between 2.5.1 and 2.6. Can anyone
confirm or deny?
-=item *BSD*
+=item BSD
Uses C<BSD::Resource::getrusage()> to determine process size. This is
pretty efficient (a lot more efficient than reading it from the
@@ -211,4 +284,6 @@
Matt Phillips E<lt>mphillips virage.comE<gt> and Mohamed Hendawi
E<lt>mhendawi virage.comE<gt>: Win32 support
+Torsten Foertsch E<lt>torsten.foertsch gmx.net<gt>: Linux::Smaps support
+
=cut
pgpPwIL7gagpy.pgp
Description: PGP signature
