Author: stevehay Date: Fri Jul 21 10:16:16 2023 New Revision: 1911170 URL: http://svn.apache.org/viewvc?rev=1911170&view=rev Log: Remove use of Linux::Smaps - it is slow and uses wrong statistics anyway
Patch by Zefram <zef...@fysh.org> on CPAN RT#93757 Modified: perl/Apache-SizeLimit/trunk/Makefile.PL perl/Apache-SizeLimit/trunk/README perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm perl/Apache-SizeLimit/trunk/t/apache/all.t perl/Apache-SizeLimit/trunk/t/apache2/all.t perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm Modified: perl/Apache-SizeLimit/trunk/Makefile.PL URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/Makefile.PL?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/Makefile.PL (original) +++ perl/Apache-SizeLimit/trunk/Makefile.PL Fri Jul 21 10:16:16 2023 @@ -23,9 +23,6 @@ else { unless ( $ARGV[0] eq '--dist' || $ENV{MOD_PERL_2_BUILD} ) { if ( $Config{'osname'} eq 'linux' ) { $prereqs{'Linux::Pid'} = 0; - if ( -e '/proc/self/smaps' ) { - $prereqs{'Linux::Smaps'} = 0; - } } elsif ( $Config{'osname'} =~ /(bsd|aix)/i ) { $prereqs{'BSD::Resource'} = 0; Modified: perl/Apache-SizeLimit/trunk/README URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/README?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/README (original) +++ perl/Apache-SizeLimit/trunk/README Fri Jul 21 10:16:16 2023 @@ -140,77 +140,6 @@ PER-PLATFORM BEHAVIOR "Apache2::SizeLimit->set_check_interval()" to reduce how often this read happens. - As of linux 2.6, /proc/self/statm does not report the amount of memory - shared by the copy-on-write mechanism as shared memory. This means that - decisions made based on shared memory as reported by that interface are - inherently wrong. - - However, as of the 2.6.14 release of the kernel, there is - /proc/self/smaps entry for each process. /proc/self/smaps reports - various sizes for each memory segment of a process and allows us to - count the amount of shared memory correctly. - - If "Apache2::SizeLimit" detects a kernel that supports /proc/self/smaps - and the "Linux::Smaps" module is installed it will use that module - instead of /proc/self/statm. - - Reading /proc/self/smaps is expensive compared to /proc/self/statm. It - must look at each page table entry of a process. Further, on - multiprocessor systems the access is synchronized with spinlocks. Again, - you might consider using "Apache2::SizeLimit->set_check_interval()". - - Copy-on-write and Shared Memory - The following example shows the effect of copy-on-write: - - <Perl> - require Apache2::SizeLimit; - package X; - use strict; - use Apache2::Const -compile => qw(OK); - - my $x = "a" x (1024*1024); - - sub handler { - my $r = shift; - my ($size, $shared) = $Apache2::SizeLimit->_check_size(); - $x =~ tr/a/b/; - my ($size2, $shared2) = $Apache2::SizeLimit->_check_size(); - $r->content_type('text/plain'); - $r->print("1: size=$size shared=$shared\n"); - $r->print("2: size=$size2 shared=$shared2\n"); - return OK; - } - </Perl> - - <Location /X> - SetHandler modperl - PerlResponseHandler X - </Location> - - The parent Apache process allocates memory for the string in $x. The - "tr"-command then overwrites all "a" with "b" if the handler is called - with an argument. This write is done in place, thus, the process size - doesn't change. Only $x is not shared anymore by means of copy-on-write - between the parent and the child. - - If /proc/self/smaps is available curl shows: - - r2@s93:~/work/mp2> curl http://localhost:8181/X - 1: size=13452 shared=7456 - 2: size=13452 shared=6432 - - Shared memory has lost 1024 kB. The process' overall size remains - unchanged. - - Without /proc/self/smaps it says: - - r2@s93:~/work/mp2> curl http://localhost:8181/X - 1: size=13052 shared=3628 - 2: size=13052 shared=3636 - - One can see the kernel lies about the shared memory. It simply doesn't - count copy-on-write pages as shared. - solaris 2.6 and above For solaris we simply retrieve the size of /proc/self/as, which contains the address-space image of the process, and convert to KB. Shared memory @@ -275,7 +204,6 @@ DEPRECATED APIS * $Apache2::SizeLimit::MIN_SHARE_SIZE * $Apache2::SizeLimit::MAX_UNSHARED_SIZE * $Apache2::SizeLimit::CHECK_EVERY_N_REQUESTS - * $Apache2::SizeLimit::USE_SMAPS Direct use of these globals is deprecated, but will continue to work for the foreseeable future. Modified: perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm (original) +++ perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm Fri Jul 21 10:16:16 2023 @@ -34,7 +34,6 @@ use Apache::SizeLimit::Core qw( $MIN_SHARE_SIZE $CHECK_EVERY_N_REQUESTS $START_TIME - $USE_SMAPS $VERSION $REQUEST_COUNT ); @@ -323,78 +322,6 @@ are worried about performance, you can c Apache::SizeLimit->set_check_interval() >> to reduce how often this read happens. -As of linux 2.6, F</proc/self/statm> does not report the amount of -memory shared by the copy-on-write mechanism as shared memory. This -means that decisions made based on shared memory as reported by that -interface are inherently wrong. - -However, as of the 2.6.14 release of the kernel, there is -F</proc/self/smaps> entry for each process. F</proc/self/smaps> -reports various sizes for each memory segment of a process and allows -us to count the amount of shared memory correctly. - -If C<Apache::SizeLimit> detects a kernel that supports -F</proc/self/smaps> and the C<Linux::Smaps> module is installed it -will use that module instead of F</proc/self/statm>. - -Reading F</proc/self/smaps> is expensive compared to -F</proc/self/statm>. It must look at each page table entry of a -process. Further, on multiprocessor systems the access is -synchronized with spinlocks. Again, you might consider using C<< -Apache::SizeLimit->set_check_interval() >>. - -=head3 Copy-on-write and Shared Memory - -The following example shows the effect of copy-on-write: - - <Perl> - require Apache::SizeLimit; - package X; - use strict; - use Apache::Constants qw(OK); - - my $x = "a" x (1024*1024); - - sub handler { - my $r = shift; - my ($size, $shared) = $Apache::SizeLimit->_check_size(); - $x =~ tr/a/b/; - my ($size2, $shared2) = $Apache::SizeLimit->_check_size(); - $r->content_type('text/plain'); - $r->print("1: size=$size shared=$shared\n"); - $r->print("2: size=$size2 shared=$shared2\n"); - return OK; - } - </Perl> - - <Location /X> - SetHandler modperl - PerlResponseHandler X - </Location> - -The parent Apache process allocates memory for the string in -C<$x>. The C<tr>-command then overwrites all "a" with "b" if the -handler is called with an argument. This write is done in place, thus, -the process size doesn't change. Only C<$x> is not shared anymore by -means of copy-on-write between the parent and the child. - -If F</proc/self/smaps> is available curl shows: - - r2@s93:~/work/mp2> curl http://localhost:8181/X - 1: size=13452 shared=7456 - 2: size=13452 shared=6432 - -Shared memory has lost 1024 kB. The process' overall size remains unchanged. - -Without F</proc/self/smaps> it says: - - r2@s93:~/work/mp2> curl http://localhost:8181/X - 1: size=13052 shared=3628 - 2: size=13052 shared=3636 - -One can see the kernel lies about the shared memory. It simply doesn't -count copy-on-write pages as shared. - =head2 solaris 2.6 and above For solaris we simply retrieve the size of F</proc/self/as>, which @@ -475,8 +402,6 @@ memory size limits: =item * $Apache::SizeLimit::CHECK_EVERY_N_REQUESTS -=item * $Apache::SizeLimit::USE_SMAPS - =back Direct use of these globals is deprecated, but will continue to work Modified: perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm (original) +++ perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm Fri Jul 21 10:16:16 2023 @@ -24,7 +24,6 @@ use Exporter; use vars qw( $VERSION $REQUEST_COUNT - $USE_SMAPS $MAX_PROCESS_SIZE $MAX_UNSHARED_SIZE @@ -41,7 +40,6 @@ use vars qw( @EXPORT_OK = qw( $VERSION $REQUEST_COUNT - $USE_SMAPS $MAX_PROCESS_SIZE $MAX_UNSHARED_SIZE $MIN_SHARE_SIZE @@ -140,17 +138,8 @@ BEGIN { } elsif ($Config{'osname'} eq 'linux') { _load('Linux::Pid'); - *_platform_getppid = \&_linux_getppid; - - if (eval { require Linux::Smaps && Linux::Smaps->new($$) }) { - $USE_SMAPS = 1; - *_platform_check_size = \&_linux_smaps_size_check; - } - else { - $USE_SMAPS = 0; - *_platform_check_size = \&_linux_size_check; - } + *_platform_check_size = \&_linux_size_check; } elsif ($Config{'osname'} =~ /(?:bsd|aix)/i) { # on OSX, getrusage() is returning 0 for proc & shared size. @@ -170,17 +159,6 @@ BEGIN { } } -sub _linux_smaps_size_check { - my $class = shift; - - return $class->_linux_size_check() unless $USE_SMAPS; - - my $s = Linux::Smaps->new($$)->all; - return ($s->size, - $s->shared_clean + $s->shared_dirty, - $s->private_clean + $s->private_dirty); -} - sub _linux_size_check { my $class = shift; Modified: perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm (original) +++ perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm Fri Jul 21 10:16:16 2023 @@ -42,7 +42,6 @@ use Apache::SizeLimit::Core qw( $MIN_SHARE_SIZE $CHECK_EVERY_N_REQUESTS $START_TIME - $USE_SMAPS $VERSION $REQUEST_COUNT ); @@ -326,78 +325,6 @@ are worried about performance, you can c Apache2::SizeLimit->set_check_interval() >> to reduce how often this read happens. -As of linux 2.6, F</proc/self/statm> does not report the amount of -memory shared by the copy-on-write mechanism as shared memory. This -means that decisions made based on shared memory as reported by that -interface are inherently wrong. - -However, as of the 2.6.14 release of the kernel, there is -F</proc/self/smaps> entry for each process. F</proc/self/smaps> -reports various sizes for each memory segment of a process and allows -us to count the amount of shared memory correctly. - -If C<Apache2::SizeLimit> detects a kernel that supports -F</proc/self/smaps> and the C<Linux::Smaps> module is installed it -will use that module instead of F</proc/self/statm>. - -Reading F</proc/self/smaps> is expensive compared to -F</proc/self/statm>. It must look at each page table entry of a -process. Further, on multiprocessor systems the access is -synchronized with spinlocks. Again, you might consider using C<< -Apache2::SizeLimit->set_check_interval() >>. - -=head3 Copy-on-write and Shared Memory - -The following example shows the effect of copy-on-write: - - <Perl> - require Apache2::SizeLimit; - package X; - use strict; - use Apache2::Const -compile => qw(OK); - - my $x = "a" x (1024*1024); - - sub handler { - my $r = shift; - my ($size, $shared) = $Apache2::SizeLimit->_check_size(); - $x =~ tr/a/b/; - my ($size2, $shared2) = $Apache2::SizeLimit->_check_size(); - $r->content_type('text/plain'); - $r->print("1: size=$size shared=$shared\n"); - $r->print("2: size=$size2 shared=$shared2\n"); - return OK; - } - </Perl> - - <Location /X> - SetHandler modperl - PerlResponseHandler X - </Location> - -The parent Apache process allocates memory for the string in -C<$x>. The C<tr>-command then overwrites all "a" with "b" if the -handler is called with an argument. This write is done in place, thus, -the process size doesn't change. Only C<$x> is not shared anymore by -means of copy-on-write between the parent and the child. - -If F</proc/self/smaps> is available curl shows: - - r2@s93:~/work/mp2> curl http://localhost:8181/X - 1: size=13452 shared=7456 - 2: size=13452 shared=6432 - -Shared memory has lost 1024 kB. The process' overall size remains unchanged. - -Without F</proc/self/smaps> it says: - - r2@s93:~/work/mp2> curl http://localhost:8181/X - 1: size=13052 shared=3628 - 2: size=13052 shared=3636 - -One can see the kernel lies about the shared memory. It simply doesn't -count copy-on-write pages as shared. - =head2 solaris 2.6 and above For solaris we simply retrieve the size of F</proc/self/as>, which @@ -478,8 +405,6 @@ memory size limits: =item * $Apache2::SizeLimit::CHECK_EVERY_N_REQUESTS -=item * $Apache2::SizeLimit::USE_SMAPS - =back Direct use of these globals is deprecated, but will continue to work Modified: perl/Apache-SizeLimit/trunk/t/apache/all.t URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/apache/all.t?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/t/apache/all.t (original) +++ perl/Apache-SizeLimit/trunk/t/apache/all.t Fri Jul 21 10:16:16 2023 @@ -15,9 +15,6 @@ sub my_need { if ( $Config{'osname'} eq 'linux' ) { $ok = need_module('Linux::Pid'); - if ( -e '/proc/self/smaps' ) { - $ok &= need_module('Linux::Smaps'); - } } elsif ( $Config{'osname'} =~ /(bsd|aix)/i ) { $ok &= need_module('BSD::Resource'); Modified: perl/Apache-SizeLimit/trunk/t/apache2/all.t URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/apache2/all.t?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/t/apache2/all.t (original) +++ perl/Apache-SizeLimit/trunk/t/apache2/all.t Fri Jul 21 10:16:16 2023 @@ -15,9 +15,6 @@ sub my_need { if ( $Config{'osname'} eq 'linux' ) { $ok = need_module('Linux::Pid'); - if ( -e '/proc/self/smaps' ) { - $ok &= need_module('Linux::Smaps'); - } } elsif ( $Config{'osname'} =~ /(bsd|aix)/i ) { $ok &= need_module('BSD::Resource'); Modified: perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm (original) +++ perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm Fri Jul 21 10:16:16 2023 @@ -30,12 +30,8 @@ sub handler { cmp_ok( $size, '>', 0, 'proc size is reported > 0' ); { - # test with USE_SMAPS=0 - my $smaps = $Apache::SizeLimit::USE_SMAPS; - $Apache::SizeLimit::USE_SMAPS = 0; my ( $size, $shared ) = Apache::SizeLimit->_check_size(); cmp_ok( $size, '>', 0, 'proc size is reported > 0' ); - $Apache::SizeLimit::USE_SMAPS = $smaps; } SKIP: Modified: perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm?rev=1911170&r1=1911169&r2=1911170&view=diff ============================================================================== --- perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm (original) +++ perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm Fri Jul 21 10:16:16 2023 @@ -30,12 +30,8 @@ sub handler { cmp_ok( $size, '>', 0, 'proc size is reported > 0' ); { - # test with USE_SMAPS=0 - my $smaps = $Apache2::SizeLimit::USE_SMAPS; - $Apache2::SizeLimit::USE_SMAPS = 0; my ( $size, $shared ) = Apache2::SizeLimit->_check_size(); cmp_ok( $size, '>', 0, 'proc size is reported > 0' ); - $Apache2::SizeLimit::USE_SMAPS = $smaps; } SKIP: