Re: mod_perl DSO leaking on restart?
Doug, I'm confused as to how you managed to *not* leak when I'm still leaking. I've tried these tests on both a Solaris 2.7 system and a Linux 7.1. Here is a summary of what I do: I build Perl ./Configure -des -Uusemymalloc -Dprefix=$(echo ~/test/prefix) -Dcc=gcc make make test make install I build Apache ~/test/prefix/bin/perl Makefile.PL USE_DSO=1 EVERYTHING=1 \ USE_APACI=1 APACHE_PREFIX=$(echo ~/test/prefix/apache) \ APACHE_SRC=../apache_1.3.19 \ APACI_ARGS='--enable-module=all --enable-shared=max' \ DO_HTTPD=1 make make test I now run the test: make start_httpd_fork ps -o 'pid,vsz,comm' -p $(cat t/logs/httpd.pid ) kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,vsz,comm' -p $(cat t/logs/httpd.pid ) kill -USR1 $(cat t/logs/httpd.pid ) etc. The virtual size grows each time, although by different amounts in Linux and Solaris. Both are around 4Mb. So what are you doing differently? Let me know so I can do the same thing. The Perl I'm using is 5.6.1 and the modperl is modperl_20010614113010.tar.gz. -P On Fri, 15 Jun 2001, Paul G. Weiss wrote: Don't be so willing to bet. Still leaking. I did as you said and just rebuilt Perl and mod_perl but didn't bother to rebuild all the Perl modules (I would have done so had I been successful here). Here's what I see: make start_httpd_fork ../apache_1.3.19/src/httpd -f `pwd`/t/conf/httpd.conf -d `pwd`/t ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 15528 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 20016 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 24544 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 27224 ../apache_1.3.19/src/httpd make kill_httpd kill `cat t/logs/httpd.pid` rm -f t/logs/httpd.pid rm -f t/logs/error_log On Thu, 14 Jun 2001, Doug MacEachern wrote: On Fri, 15 Jun 2001, Paul G. Weiss wrote: alignbytes=8, usemymalloc=y, prototype=define ^ ok, here's why i kept asking for perl -V. i don't see Perl's malloc.c ever release its memory pool. when usemymalloc=y, free() only puts memory back into Perl's pool for use by other malloc()'s. i don't see a function to destroy this pool when perl cleans itself up. willing to bet if you rebuild Perl with: Configure -des -Uusemymalloc ... and then rebuild mod_perl, the leaks will go away.
Re: mod_perl DSO leaking on restart?
Now I'm really confused. I built the whole thing statically and it still leaks: the static build (using the same Perl): ~/test/prefix/bin/perl Makefile.PL EVERYTHING=1 \ APACHE_PREFIX=$(echo ~/test/prefix/apache) \ APACHE_SRC=../apache_1.3.19 DO_HTTPD=1 Now it still leaks 520K per restart: make start_httpd_fork ../apache_1.3.19/src/httpd -f `pwd`/t/conf/httpd.conf -d `pwd`/t ps -o pid,vsz,comm -p $(cat t/logs/httpd.pid ) PID VSZ COMMAND 4877 7856 httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o pid,vsz,comm -p $(cat t/logs/httpd.pid ) PID VSZ COMMAND 4877 8488 httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o pid,vsz,comm -p $(cat t/logs/httpd.pid ) PID VSZ COMMAND 4877 9108 httpd This is on Redhat 7.1 (not Linux 7.1 as I said below). I didn't bother trying it on Solaris. You don't suppose building with APACHE_PREFIX could have anything to do with it, do you? I haven't heard of any leaks with a static build. -P On Sun, 17 Jun 2001, Paul G. Weiss wrote: Doug, I'm confused as to how you managed to *not* leak when I'm still leaking. I've tried these tests on both a Solaris 2.7 system and a Linux 7.1. Here is a summary of what I do: I build Perl ./Configure -des -Uusemymalloc -Dprefix=$(echo ~/test/prefix) -Dcc=gcc make make test make install I build Apache ~/test/prefix/bin/perl Makefile.PL USE_DSO=1 EVERYTHING=1 \ USE_APACI=1 APACHE_PREFIX=$(echo ~/test/prefix/apache) \ APACHE_SRC=../apache_1.3.19 \ APACI_ARGS='--enable-module=all --enable-shared=max' \ DO_HTTPD=1 make make test I now run the test: make start_httpd_fork ps -o 'pid,vsz,comm' -p $(cat t/logs/httpd.pid ) kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,vsz,comm' -p $(cat t/logs/httpd.pid ) kill -USR1 $(cat t/logs/httpd.pid ) etc. The virtual size grows each time, although by different amounts in Linux and Solaris. Both are around 4Mb. So what are you doing differently? Let me know so I can do the same thing. The Perl I'm using is 5.6.1 and the modperl is modperl_20010614113010.tar.gz. -P On Fri, 15 Jun 2001, Paul G. Weiss wrote: Don't be so willing to bet. Still leaking. I did as you said and just rebuilt Perl and mod_perl but didn't bother to rebuild all the Perl modules (I would have done so had I been successful here). Here's what I see: make start_httpd_fork ../apache_1.3.19/src/httpd -f `pwd`/t/conf/httpd.conf -d `pwd`/t ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 15528 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 20016 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 24544 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 27224 ../apache_1.3.19/src/httpd make kill_httpd kill `cat t/logs/httpd.pid` rm -f t/logs/httpd.pid rm -f t/logs/error_log On Thu, 14 Jun 2001, Doug MacEachern wrote: On Fri, 15 Jun 2001, Paul G. Weiss wrote: alignbytes=8, usemymalloc=y, prototype=define ^ ok, here's why i kept asking for perl -V. i don't see Perl's malloc.c ever release its memory pool. when usemymalloc=y, free() only puts memory back into Perl's pool for use by other malloc()'s. i don't see a function to destroy this pool when perl cleans itself up. willing to bet if you rebuild Perl with: Configure -des -Uusemymalloc ... and then rebuild mod_perl, the leaks will go away.
Re: mod_perl DSO leaking on restart?
Don't be so willing to bet. Still leaking. I did as you said and just rebuilt Perl and mod_perl but didn't bother to rebuild all the Perl modules (I would have done so had I been successful here). Here's what I see: make start_httpd_fork ../apache_1.3.19/src/httpd -f `pwd`/t/conf/httpd.conf -d `pwd`/t ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 15528 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 20016 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 24544 ../apache_1.3.19/src/httpd kill -USR1 $(cat t/logs/httpd.pid ) ps -o 'pid,ppid,vsz,comm' -p $(cat t/logs/httpd.pid ) PID PPID VSZ COMMAND 28802 1 27224 ../apache_1.3.19/src/httpd make kill_httpd kill `cat t/logs/httpd.pid` rm -f t/logs/httpd.pid rm -f t/logs/error_log On Thu, 14 Jun 2001, Doug MacEachern wrote: On Fri, 15 Jun 2001, Paul G. Weiss wrote: alignbytes=8, usemymalloc=y, prototype=define ^ ok, here's why i kept asking for perl -V. i don't see Perl's malloc.c ever release its memory pool. when usemymalloc=y, free() only puts memory back into Perl's pool for use by other malloc()'s. i don't see a function to destroy this pool when perl cleans itself up. willing to bet if you rebuild Perl with: Configure -des -Uusemymalloc ... and then rebuild mod_perl, the leaks will go away.
Re: mod_perl DSO leaking on restart?
On Wed, 25 Apr 2001, Philip Mak wrote: On Thu, 26 Apr 2001, Stas Bekman wrote: There is also the strange case of mod_perl leaking memory on graceful restarts when compiled as DSO. But I don't feel like getting into this one quite yet. Hmm. My httpd was using 20 MB. I did apachectl graceful ten times, and the usage jumped to 24 MB. Then I did apachectl graceful another ten times, and the usage jumped to 29 MB. I guess that's the reason (or one of them) that my httpd grows bigger and bigger as time passes. My mod_perl is a DSO (I run two copies of httpd, one without mod_perl and one with; I set it up as a DSO since this way I only need one executable). Should I recompile it statically linked? repeat 1.21_01 had two dso fixes, one to close all .so's opened by DynaLoader and one to call perl_shutdown(), both of which were large leaks. with 1.25_01-dev and Perl 5.6.1 i see a 4k growth on the first kill -USR1 and no change after that. what is your perl -V and mod_perl version? /repeat if people are seeing leaks on restart using Perl 5.005_03 i am not surprised, 5.6.1 plugs a great many leaks.
Re: mod_perl DSO leaking on restart?
On Thu, 14 Jun 2001, Doug MacEachern wrote: repeat 1.21_01 had two dso fixes, one to close all .so's opened by DynaLoader and one to call perl_shutdown(), both of which were large leaks. with 1.25_01-dev and Perl 5.6.1 i see a 4k growth on the first kill -USR1 and no change after that. what is your perl -V and mod_perl version? /repeat if people are seeing leaks on restart using Perl 5.005_03 i am not surprised, 5.6.1 plugs a great many leaks. Perl 5.005_03, mod_perl 1.25. I have since fixed the memory leak problem by recompiling mod_perl so that it is statically linked to Apache instead of as a DSO. -Philip Mak ([EMAIL PROTECTED])
Re: mod_perl DSO leaking on restart?
On Thu, 14 Jun 2001, Paul G. Weiss wrote: Sad to say, I'm not as fortunate as you. I'm leaking ~4.4Mb which each HUP (I haven't tried USR1). I'm also using Perl 5.6.1 and 1.25_01-dev (the CVS snapshot modperl_20010614113010.tar.gz). I'm doing a PerlRequire if that is significant. One more thing - I'm building mod_perl using apxs. I don't know if that makes a difference. i am surprised to see, it does make a difference. i built with: perl Makefile.PL USE_DSO=1 EVERYTHING=1 - no leaks. perl Makefile.PL USE_APXS=1 WITH_APXS=... EVERYTHING=1 - leaks. i haven't looked into it yet. can you try with USE_DSO=1 ? if you're still leaking please post your 'perl -V' and the full list of Makefile.PL options you're using.
Re: mod_perl DSO leaking on restart?
On Fri, 15 Jun 2001, Paul G. Weiss wrote: alignbytes=8, usemymalloc=y, prototype=define ^ ok, here's why i kept asking for perl -V. i don't see Perl's malloc.c ever release its memory pool. when usemymalloc=y, free() only puts memory back into Perl's pool for use by other malloc()'s. i don't see a function to destroy this pool when perl cleans itself up. willing to bet if you rebuild Perl with: Configure -des -Uusemymalloc ... and then rebuild mod_perl, the leaks will go away.
Re: mod_perl DSO leaking on restart?
Sad to say, I'm not as fortunate as you. I'm leaking ~4.4Mb which each HUP (I haven't tried USR1). I'm also using Perl 5.6.1 and 1.25_01-dev (the CVS snapshot modperl_20010614113010.tar.gz). I'm doing a PerlRequire if that is significant. One more thing - I'm building mod_perl using apxs. I don't know if that makes a difference. -Paul On Thu, 14 Jun 2001, Doug MacEachern wrote: On Wed, 25 Apr 2001, Philip Mak wrote: On Thu, 26 Apr 2001, Stas Bekman wrote: There is also the strange case of mod_perl leaking memory on graceful restarts when compiled as DSO. But I don't feel like getting into this one quite yet. Hmm. My httpd was using 20 MB. I did apachectl graceful ten times, and the usage jumped to 24 MB. Then I did apachectl graceful another ten times, and the usage jumped to 29 MB. I guess that's the reason (or one of them) that my httpd grows bigger and bigger as time passes. My mod_perl is a DSO (I run two copies of httpd, one without mod_perl and one with; I set it up as a DSO since this way I only need one executable). Should I recompile it statically linked? repeat 1.21_01 had two dso fixes, one to close all .so's opened by DynaLoader and one to call perl_shutdown(), both of which were large leaks. with 1.25_01-dev and Perl 5.6.1 i see a 4k growth on the first kill -USR1 and no change after that. what is your perl -V and mod_perl version? /repeat if people are seeing leaks on restart using Perl 5.005_03 i am not surprised, 5.6.1 plugs a great many leaks.
Re: mod_perl DSO leaking on restart?
Still leaking I'm afraid, by roughly the same amount. I'm posting the files you asked for. Perhaps it has something to do with the fact that both the Perl and Apache are installed in non-standard places, or that I'm also using mod_ssl. Here goes: perl -V: Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=solaris, osvers=2.7, archname=sun4-solaris uname='sunos us2 5.7 generic_106541-12 sun4u sparc sunw,ultrasparc-iii-engine ' config_args='-e -Ubincompat5005 -Dprefix=/vol1/home/tecadmin/envs/env_nothreads -Doptimize=-O -g -Dcc=gcc -U installusrbinperl -Dloclibpth=/vol1/home/tecadmin/envs/env_nothreads/lib /usr/local/lib -Dlocincpth=/vol1/home/t ecadmin/envs/env_nothreads/include /usr/local/include -ds' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='gcc', ccflags ='-DDEBUGGING -fno-strict-aliasing -I/vol1/home/tecadmin/envs/env_nothreads/include -I/usr /local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O -g', cppflags='-DDEBUGGING -fno-strict-aliasing -I/vol1/home/tecadmin/envs/env_nothreads/include -I/usr/local/inc lude' ccversion='', gccversion='2.95.2 19991024 (release)', gccosandvers='solaris2.7' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, usemymalloc=y, prototype=define Linker and Libraries: ld='gcc', ldflags =' -L/vol1/home/tecadmin/envs/env_nothreads/lib -L/usr/local/lib ' libpth=/vol1/home/tecadmin/envs/env_nothreads/lib /usr/local/lib /usr/lib /usr/ccs/lib libs=-lsocket -lnsl -ldb -ldl -lm -lc perllibs=-lsocket -lnsl -ldl -lm -lc libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' ' cccdlflags='-fPIC', lddlflags='-G -L/vol1/home/tecadmin/envs/env_nothreads/lib -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: DEBUGGING USE_LARGE_FILES Built under solaris Compiled at Jun 13 2001 12:52:50 %ENV: PERL5LIB=/vol1/home/pweiss/perllib @INC: /vol1/home/pweiss/perllib /vol1/home/tecadmin/envs/env_nothreads/lib/perl5/5.6.1/sun4-solaris /vol1/home/tecadmin/envs/env_nothreads/lib/perl5/5.6.1 /vol1/home/tecadmin/envs/env_nothreads/lib/perl5/site_perl/5.6.1/sun4-solaris /vol1/home/tecadmin/envs/env_nothreads/lib/perl5/site_perl/5.6.1 /vol1/home/tecadmin/envs/env_nothreads/lib/perl5/site_perl . Here is how I do Makefile.PL: LIBS_SHLIB=$(gcc -print-libgcc-file-name) CFLAGS='-O -g' \ perl Makefile.PL SSL_BASE=/vol1/home/tecadmin/envs/env_nothreads \ USE_APACI=1 USE_DSO=1 EVERYTHING=1 \ APACHE_PREFIX=/vol1/home/tecadmin/envs/env_nothreads/apache \ APACHE_SOURCE=../apache_1.3.19 \ APACI_ARGS='--enable-module=all --enable-shared=max' DO_HTTPD=1 -Paul P.S. By the way, the CFLAGS='-O -g' appears to have no affect at all. I can't seem to get Apache compiled with symbols. On Thu, 14 Jun 2001, Doug MacEachern wrote: On Thu, 14 Jun 2001, Paul G. Weiss wrote: Sad to say, I'm not as fortunate as you. I'm leaking ~4.4Mb which each HUP (I haven't tried USR1). I'm also using Perl 5.6.1 and 1.25_01-dev (the CVS snapshot modperl_20010614113010.tar.gz). I'm doing a PerlRequire if that is significant. One more thing - I'm building mod_perl using apxs. I don't know if that makes a difference. i am surprised to see, it does make a difference. i built with: perl Makefile.PL USE_DSO=1 EVERYTHING=1 - no leaks. perl Makefile.PL USE_APXS=1 WITH_APXS=... EVERYTHING=1 - leaks. i haven't looked into it yet. can you try with USE_DSO=1 ? if you're still leaking please post your 'perl -V' and the full list of Makefile.PL options you're using.
Re: mod_perl DSO leaking on restart?
On Wed, Apr 25, 2001 at 10:40:12PM -0400, Philip Mak wrote: On Thu, 26 Apr 2001, Stas Bekman wrote: There is also the strange case of mod_perl leaking memory on graceful restarts when compiled as DSO. But I don't feel like getting into this one quite yet. Hmm. My httpd was using 20 MB. I did apachectl graceful ten times, and the usage jumped to 24 MB. Then I did apachectl graceful another ten times, and the usage jumped to 29 MB. I guess that's the reason (or one of them) that my httpd grows bigger and bigger as time passes. My mod_perl is a DSO (I run two copies of httpd, one without mod_perl and one with; I set it up as a DSO since this way I only need one executable). Should I recompile it statically linked? I had the same problems, after linking modperl statically into httpd the leaks disappeared. -- Jens-Uwe Mager HELIOS Software GmbH Steinriede 3 30827 Garbsen Germany Phone: +49 5131 709320 FAX:+49 5131 709325 Internet: [EMAIL PROTECTED]
Re: mod_perl DSO leaking on restart?
Same here. I did a few graceful restarts and the usage jumped to 80M from 10M... Then I apachectl stop and apachectl start to get a clean restart, it works fine. I am new to this list. Maybe this problem has been addressed... At 10:40 PM 2001/4/25 -0400, Philip Mak wrote: On Thu, 26 Apr 2001, Stas Bekman wrote: There is also the strange case of mod_perl leaking memory on graceful restarts when compiled as DSO. But I don't feel like getting into this one quite yet. Hmm. My httpd was using 20 MB. I did apachectl graceful ten times, and the usage jumped to 24 MB. Then I did apachectl graceful another ten times, and the usage jumped to 29 MB. I guess that's the reason (or one of them) that my httpd grows bigger and bigger as time passes. My mod_perl is a DSO (I run two copies of httpd, one without mod_perl and one with; I set it up as a DSO since this way I only need one executable). Should I recompile it statically linked? -Philip Mak ([EMAIL PROTECTED])
