Re: Memory leak on reload when the 'Pg' driver is preloaded
KGM == Keith G Murphy [EMAIL PROTECTED] writes: KGM When using a modular mod_perl, I get a huge leak if I preload the 'Pg' KGM driver in my startup perl script thus: Hmmm. Interesting theory. I shall have to investigate it. I also see a multi-megabyte memory leak in my app when DBD::Pg is preloaded, but I didn't track it down to just that module. Does it happen for you for any perl module that loads another XS compiled (ie, .so) file in addition to the .pm file? I'm on FreeBSD. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D.Khera Communications, Inc. Internet: [EMAIL PROTECTED] Rockville, MD +1-240-453-8497 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
Re: Memory leak on reload when the 'Pg' driver is preloaded
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
OK, it gets weirder. The following script produces the leak. If I
comment out the install_driver line, I get a big old segfault! Same
if I comment out the Apache::DBI line in addition. This works with
plain apache, or apache-ssl.
#!/usr/bin/perl
use strict;
use Apache::Status ();
use Apache::DBI ();
DBI-install_driver('Pg');
1; #return true value
I looked into it with the following setup:
apache 1.3.26-0woody1
libapache-mod-perl 1.27-2
postgresql 7.2.1-2woody2
There was a Test.pm module handling all requests for /. It opened a
connection to the database and fetched a couple of rows.
With DBI-install_driver('Pg'), the Apache processes stayed at 8.4 MB. I
bombarded the Apache with ApacheBench and multiple SIGUSR1 but I found
no memory leaks. The process sizes did not change.
If I comment it all out including the Apache::Status line, I get
neither the segfault or the leak. (Well, small leak, a few bytes).
Without the install_driver() in place, everything worked as well. The
Apache processes stayed at near 6 MB. No segfaults. No problems.
So the install_driver masks a bigger problem with Apache::Status?
That's just too weird. But true, I swear.
Is there anyone else out there with Debian stable w/
apache+libapache-mod-perl who can try to reproduce this simple case?
I'd really like either a confirmation or a counterexample. If the
latter, I'd chalk it up to a weirdness on my system (and probably not
even pursue it, since I've moved on to the static version). I'm
pursuing it this far so Dan J. might get some useful feedback.
I'm pretty tired at the moment, so there might have been something I
mislooked. I'll give it another look tomorrow. At work, where I don't
have any distractions :)
It's pretty odd, in my view, that the Apache processes which did not
preload the Pg driver were smaller.
Keith, could you post results to modperl list if people at debian-isp
come to a different (or to the same) solution. I'm not subscribed to
the list.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE9rxEXnksV4Ys/z5gRAqlBAJ9xKgkCmCxsg8huo/C1N0uk2XKSowCgoCdr
FdyuRGCkZvDifk5fyqfOcdo=
=im3K
-END PGP SIGNATURE-
Re: Memory leak on reload when the 'Pg' driver is preloaded
Juha-Mikko Ahonen wrote:
I looked into it with the following setup:
apache 1.3.26-0woody1
libapache-mod-perl 1.27-2
postgresql 7.2.1-2woody2
There was a Test.pm module handling all requests for /. It opened a
connection to the database and fetched a couple of rows.
With DBI-install_driver('Pg'), the Apache processes stayed at 8.4 MB. I
bombarded the Apache with ApacheBench and multiple SIGUSR1 but I found
no memory leaks. The process sizes did not change.
The interesting thing I see here is that you are using a different
libapache-mod-perl version. I'm using the one from stable. I hesitate
to install yours (from unstable), since apt-get wants to upgrade all of
perl (a whole *bunch* of shit), and I'm not keen on the idea of
permanently tracking unstable with something as complex as Perl and its
dependents. I mean, crap, it even wants to upgrade libc6. Last time I
tried, downgrading libc6 was not fun. I'm guessing you might find it
difficult to downgrade as well, since you probably have Perl 5.8. Any
ideas?
I'm pretty tired at the moment, so there might have been something I
mislooked. I'll give it another look tomorrow. At work, where I don't
have any distractions :)
I appreciate you looking at this. But it's mainly for the maintainer at
this point...
It's pretty odd, in my view, that the Apache processes which did not
preload the Pg driver were smaller.
Why would that be odd? Seems intuitively right to me.
Keith, could you post results to modperl list if people at debian-isp
come to a different (or to the same) solution. I'm not subscribed to
the list.
Will do, for sure.
Memory leak on reload when the 'Pg' driver is preloaded
Since memory leaks seem to be the topic du jour, I wondered if anyone
else had seen this one:
When using a modular mod_perl, I get a huge leak if I preload the 'Pg'
driver in my startup perl script thus:
#!/usr/bin/perl
use strict;
use Apache::Status ();
use Apache::DBI ();
DBI-install_driver('Pg');
1; #return true value
The leak happens when you send SIGUSR1 to the parent. Every time you do
it, you get more memory eaten (multiple megabytes), both in the parent
and (new) children.
It happens with the stock apache-ssl that comes from Debian stable.
Could well be a Debian-specific problem.
Using Debian's static-mod_perled apache-perl eliminates the problem.
Re: Memory leak on reload when the 'Pg' driver is preloaded
Using Debian's static-mod_perled apache-perl eliminates the problem. Do you mean you are using the 'so' version that comes with Debian? You should be using the static build of apache/mod_perl
Re: Memory leak on reload when the 'Pg' driver is preloaded
Stathy G. Touloumis wrote: Using Debian's static-mod_perled apache-perl eliminates the problem. Do you mean you are using the 'so' version that comes with Debian? Yes, in the case that failed. The package is called 'libapache-mod-perl'. You should be using the static build of apache/mod_perl You're telling me. :-( By should, do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? Might it be related to the Perl-malloc problem cited here: http://apache.perl.org/docs/1.0/guide/install.html#When_DSO_can_be_Used If either of the above or true, seems like Debian needs to quit distributing libapache-mod-perl. Thanks.
Re: Memory leak on reload when the 'Pg' driver is preloaded
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wednesday 16 October 2002 20:25, Keith G. Murphy wrote: By should, do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? Yes. The problems with DSO mod_perl are well known. It works perfectly when build staticly into Apache, though. Install apache-perl package instead of apache and libapache-mod-perl. Everything works with it as it would on normal Apache. After migrating to apache-perl, remove the LoadModule for mod_perl from your Apache configuration. Might it be related to the Perl-malloc problem cited here: http://apache.perl.org/docs/1.0/guide/install.html#When_DSO_can_be_Us ed If either of the above or true, seems like Debian needs to quit distributing libapache-mod-perl. The problem is not Debian-related, so it won't justify removing the package from Debian. -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE9raVtnksV4Ys/z5gRAt+TAJwOZXcdnVNoLWLza7NU6eOpCOB+dQCdEliU VDvG6M+qI88/RN/jj7KsCC0= =0/Bl -END PGP SIGNATURE-
Re: Memory leak on reload when the 'Pg' driver is preloaded
Hi there, On Wed, 16 Oct 2002, Keith G. Murphy wrote: do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? It's not as bad as that. Significant improvements have been made in the reliability of mod_perl as DSO and nowadays there is much less discussion about it on this list. There are still one or two dusty corners but in general thesedays I'd say try it. If it doesn't seem to give you problems then stay with it. If at first you don't succeed, try again. Then give up. No sense being a damned fool about it. (Somebody remind me who said that:) 73, Ged.
Re: Memory leak on reload when the 'Pg' driver is preloaded
Ged Haywood wrote: Hi there, On Wed, 16 Oct 2002, Keith G. Murphy wrote: do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? It's not as bad as that. Significant improvements have been made in the reliability of mod_perl as DSO and nowadays there is much less discussion about it on this list. Are you sure it's not because 'most everyone has silently given up on it? There are still one or two dusty corners but in general thesedays I'd say try it. I had not expected Debian stable to be one of the dusty corners. But I did find, upon investigating the bug reports, that there were *very* old reports about memory leaks, etc., with libapache-mod-perl. My own bug report is now 47 days old, without apparent followup. If it doesn't seem to give you problems then stay with it. If at first you don't succeed, try again. Then give up. Actually, that is what I have done already, several months ago. Seeing several reports of memory leak problems in the list made me think: Hmmm, wonder if someone else has seen this?
Re: Memory leak on reload when the 'Pg' driver is preloaded
Juha-Mikko Ahonen wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wednesday 16 October 2002 20:25, Keith G. Murphy wrote: By should, do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? Yes. The problems with DSO mod_perl are well known. It works perfectly when build staticly into Apache, though. Install apache-perl package instead of apache and libapache-mod-perl. Everything works with it as it would on normal Apache. After migrating to apache-perl, remove the LoadModule for mod_perl from your Apache configuration. I already have, months ago. Might it be related to the Perl-malloc problem cited here: http://apache.perl.org/docs/1.0/guide/install.html#When_DSO_can_be_Us ed If either of the above or true, seems like Debian needs to quit distributing libapache-mod-perl. The problem is not Debian-related, so it won't justify removing the package from Debian. Huh? I would think that would be a really good reason to yank it. If it were Debian-related, the maintainer could fix it and keep it.
RE: Memory leak on reload when the 'Pg' driver is preloaded
For what it is worth, we use a DSO mod_perl/apache that we compiled ourselves, on RedHat thought, that is very stable and does not have noticeable memory leaks and have been using it for over 3 years. Just thought I would throw that out there. -Original Message- From: Keith G. Murphy [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 16, 2002 2:02 PM To: mod_perl Mailing List Subject: Re: Memory leak on reload when the 'Pg' driver is preloaded Ged Haywood wrote: Hi there, On Wed, 16 Oct 2002, Keith G. Murphy wrote: do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? It's not as bad as that. Significant improvements have been made in the reliability of mod_perl as DSO and nowadays there is much less discussion about it on this list. Are you sure it's not because 'most everyone has silently given up on it? There are still one or two dusty corners but in general thesedays I'd say try it. I had not expected Debian stable to be one of the dusty corners. But I did find, upon investigating the bug reports, that there were *very* old reports about memory leaks, etc., with libapache-mod-perl. My own bug report is now 47 days old, without apparent followup. If it doesn't seem to give you problems then stay with it. If at first you don't succeed, try again. Then give up. Actually, that is what I have done already, several months ago. Seeing several reports of memory leak problems in the list made me think: Hmmm, wonder if someone else has seen this?
Re: Memory leak on reload when the 'Pg' driver is preloaded
Hi there, On Wed, 16 Oct 2002, Keith G. Murphy wrote: Significant improvements have been made in the reliability of mod_perl as DSO and nowadays there is much less discussion about it on this list. Are you sure it's not because 'most everyone has silently given up on it? Yes, I'm sure. Red Hat comes with Apache prebuilt for DSO I believe. I have worked on contracts for a number of organizations which use DSO mod_perl in high volume environments (hundreds of thousands of users). There are still one or two dusty corners but in general thesedays I'd say try it. I had not expected Debian stable to be one of the dusty corners. What I meant was that some (perhaps slightly unusual in some way) configurations seem to give more problems than others. For example I think that servers using SSL have perhaps been prone to troubles, but it could just as easily be that the people who experienced the problems were not careful to follow what you might say are slightly more complex than usual build procedures. 73, Ged.
Re: Memory leak on reload when the 'Pg' driver is preloaded
Ged Haywood wrote: Hi there, On Wed, 16 Oct 2002, Keith G. Murphy wrote: Significant improvements have been made in the reliability of mod_perl as DSO and nowadays there is much less discussion about it on this list. Are you sure it's not because 'most everyone has silently given up on it? Yes, I'm sure. Red Hat comes with Apache prebuilt for DSO I believe. I have worked on contracts for a number of organizations which use DSO mod_perl in high volume environments (hundreds of thousands of users). Then I take your word for it. (My question was sincere, not argumentative, by the way). There are still one or two dusty corners but in general thesedays I'd say try it. I had not expected Debian stable to be one of the dusty corners. What I meant was that some (perhaps slightly unusual in some way) configurations seem to give more problems than others. For example I think that servers using SSL have perhaps been prone to troubles, but it could just as easily be that the people who experienced the problems were not careful to follow what you might say are slightly more complex than usual build procedures. Hmmm. In my case, I replicated the problem with Debian's plain vanilla Apache as well, for the very purpose of simplification. So SSL does not seem to be involved.
Re: Memory leak on reload when the 'Pg' driver is preloaded
On Wed, Oct 16, 2002 at 02:01:33PM -0500, Keith G. Murphy wrote: Ged Haywood wrote: Hi there, On Wed, 16 Oct 2002, Keith G. Murphy wrote: do you mean that the problems with the loadable module overall are so well-known that no one in his right mind should ever use it? It's not as bad as that. Significant improvements have been made in the reliability of mod_perl as DSO and nowadays there is much less discussion about it on this list. Are you sure it's not because 'most everyone has silently given up on it? There are still one or two dusty corners but in general thesedays I'd say try it. I had not expected Debian stable to be one of the dusty corners. But I did find, upon investigating the bug reports, that there were *very* old reports about memory leaks, etc., with libapache-mod-perl. My own bug report is now 47 days old, without apparent followup. That's because I'm having an attack of real life. I haven't had time to look at my Debian bug reports in a month or so now. I don't know why the DSO is leaking in a case when the static module isn't, but it should be fixable; I just need to find five or six hours to sit down with it. -- Daniel Jacobowitz MontaVista Software Debian GNU/Linux Developer
Re: Memory leak on reload when the 'Pg' driver is preloaded
Daniel Jacobowitz wrote: On Wed, Oct 16, 2002 at 02:01:33PM -0500, Keith G. Murphy wrote: My own bug report is now 47 days old, without apparent followup. Hmmm, I probably should not have posted that. Sounds like a major whine. That's because I'm having an attack of real life. I haven't had time to look at my Debian bug reports in a month or so now. I don't know why the DSO is leaking in a case when the static module isn't, but it should be fixable; I just need to find five or six hours to sit down with it. Thanks, Dan, this really wasn't meant to be a slam-the-developer session. I really, *really* appreciate all the work you've done already. Now I feel tacky. :-( It's not like it was an obvious problem: I only got the DSO to leak when loading the Pg driver. That's pretty obscure.
Re: Memory leak on reload when the 'Pg' driver is preloaded
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Wednesday 16 October 2002 22:52, Keith G. Murphy wrote:
It's not like it was an obvious problem: I only got the DSO to leak
when loading the Pg driver. That's pretty obscure.
Have you tried to connect() without loading the Pg driver first? E.g.
$dbh = DBI-connect('dbi:Pg:dbname=foo;host=bar', 'user', 'pass');
It would help eliminate if the leak is caused by your startup script. If
it is, it may be the DBI module that's leaking memory.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE9rcZ5nksV4Ys/z5gRAjBQAJwKOUuu4B14yLe1msSNekI0mnPvdACdFMOe
TFLM0arS4L177Y+OuJMAUgM=
=GsvK
-END PGP SIGNATURE-
Re: Memory leak on reload when the 'Pg' driver is preloaded
[debian-isp readers, to recap, I'm trying to confirm a
memory-leak/segfault problem with Debian stable plus apache(-ssl) plus
libapache-mod-perl. The memory leak happens upon
/etc/init.d/apache(-ssl) reload. You can see my startup script and my
other comments below.]
Juha-Mikko Ahonen wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Wednesday 16 October 2002 22:52, Keith G. Murphy wrote:
It's not like it was an obvious problem: I only got the DSO to leak
when loading the Pg driver. That's pretty obscure.
Have you tried to connect() without loading the Pg driver first? E.g.
$dbh = DBI-connect('dbi:Pg:dbname=foo;host=bar', 'user', 'pass');
It would help eliminate if the leak is caused by your startup script. If
it is, it may be the DBI module that's leaking memory.
OK, it gets weirder. The following script produces the leak. If I
comment out the install_driver line, I get a big old segfault! Same if
I comment out the Apache::DBI line in addition. This works with plain
apache, or apache-ssl.
#!/usr/bin/perl
use strict;
use Apache::Status ();
use Apache::DBI ();
DBI-install_driver('Pg');
1; #return true value
If I comment it all out including the Apache::Status line, I get neither
the segfault or the leak. (Well, small leak, a few bytes).
So the install_driver masks a bigger problem with Apache::Status?
That's just too weird. But true, I swear.
Is there anyone else out there with Debian stable w/
apache+libapache-mod-perl who can try to reproduce this simple case?
I'd really like either a confirmation or a counterexample. If the
latter, I'd chalk it up to a weirdness on my system (and probably not
even pursue it, since I've moved on to the static version). I'm
pursuing it this far so Dan J. might get some useful feedback.
I've submitted this to debian-isp as well; maybe someone there can try it.
