experiencing Out of memory errors
We have an Apache/mod_perl application running under Windows Server 2003 that periodically experiences "Out of memory" errors (they appear in the Apache error logs) which forces Apache to restart. Our application does, in fact, use a lot of memory, and we believe that this is not due to a bug or memory leak. What we would like is to run this application in a 64 bit environment, thus allowing us to use more than the 2 GB memory that the Apache process ('httpd') is limited to (the "Out of memory" errors always occur as the memory usage of the httpd process approaches this 2 GB limit). We know that we will need to run a 64 bit version of Windows on 64 bit hardware to accomplish this. Furthermore, we know that ActiveState has a binary Perl install of a 64 bit Perl. However, we're not aware of either a 64 bit Apache (though recently I found the blackdot site - http://www.blackdot.be/ - that provides a Windows compatible binary build of 64 bit apache, though I know very little about it) or a 64 bit mod_perl. However, we're also not sure if either of these are needed to escape the 2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit Apache and/or mod_perl? Any help with accomplishing what we need would be greatly appreciated, including the possibility of hiring someone on a contract basis to help us. OS: Microsoft Windows Server 2003 Perl: v 5.8.9 ActiveState build 826 Apache 2.2.11 mod_perl 2 (not sure of the exact version, but very recent)
Re: experiencing Out of memory errors
Hi John, On Wed, 2011-01-26 at 10:17 -0500, John Deighan wrote: > However, we're also not sure if either of these are needed to escape the > 2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit > Apache and/or mod_perl? You should use the same architecture for Perl and Apache. > Any help with accomplishing what we need would be greatly appreciated, > including the possibility of hiring someone on a contract basis to > help us. Probably because of the fact that the Apache foundation does not ship 64-bit builds for Windows (which is a shame!), ActiveState does not ship a mod_perl with their 64- bit Perl. Compiling your own mod_perl would be possible but you need to use preferably the same compiler that was used for Apache + perl. If you would want to switch to StrawberryPerl 5.12.x 64-bit, you can use this pre-compiled mod_perl: http://strawberryperl.com/package/kmx/mod_perl/ -- Mike
Re: experiencing Out of memory errors
Hi Am 27.01.2011 um 12:57 schrieb Michiel Beijen : > Hi John, > > On Wed, 2011-01-26 at 10:17 -0500, John Deighan wrote: >> However, we're also not sure if either of these are needed to escape the >> 2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit >> Apache and/or mod_perl? > > You should use the same architecture for Perl and Apache. > >> Any help with accomplishing what we need would be greatly appreciated, >> including the possibility of hiring someone on a contract basis to >> help us. > > Probably because of the fact that the Apache foundation does not ship > 64-bit builds for Windows (which is a shame!), ActiveState does not ship > a mod_perl with their 64- bit Perl. Compiling your own mod_perl would be > possible but you need to use preferably the same compiler that was used > for Apache + perl. > > If you would want to switch to StrawberryPerl 5.12.x 64-bit, you can use > this pre-compiled mod_perl: > http://strawberryperl.com/package/kmx/mod_perl/ Is mod_perl compiled with MinGW or with cl.exe for windows? Seems that apxs suggest MinGW. I got the same problem with mod_ruby for windows. Best Zeno
Re: experiencing Out of memory errors
I'd forgotten about Strawberry Perl (came across it about a year ago, but never got around to trying it). One question, though: I'm thinking there's not much point to using a 64-bit version of mod_perl unless I'm using it with a 64-bit version of Apache. Where can I get a 64-bit version of Apache that will with with the mod_perl below? On 1/27/2011 6:57 AM, Michiel Beijen wrote: Hi John, ... etc. If you would want to switch to StrawberryPerl 5.12.x 64-bit, you can use this pre-compiled mod_perl: http://strawberryperl.com/package/kmx/mod_perl/ -- Mike
Re: experiencing Out of memory errors
Hi John, On Thu, January 27, 2011 15:10, John Deighan wrote: > One question, though: I'm thinking there's not much point to using a > 64-bit version of mod_perl > unless I'm using it with a 64-bit version of Apache. Where can I get a > 64-bit version of Apache that > will with with the mod_perl below? You're right, you should not use 64-bit perl with a 32-bit apache You should get the one from Blackdot probably: http://www.blackdot.be/?inc=apache/binaries -- Mike
Re: experiencing Out of memory errors
On 26 Jan 2011, at 15:17, John Deighan wrote: > What we would like is to run this application in a 64 bit environment, thus > allowing us to use more than the 2 GB memory that the Apache process ('httpd') > is limited to (the "Out of memory" errors always occur as the memory usage > of the httpd process approaches this 2 GB limit). Can I just say: WTF? 2G in an Apache? Surely there's a better way of architecting this? smime.p7s Description: S/MIME cryptographic signature
Re: experiencing Out of memory errors
On 01/27/2011 07:05 PM, Dave Hodgkinson wrote: Can I just say: WTF? 2G in an Apache? Surely there's a better way of architecting this? One thing to remember is that he's running Windows which doesn't have Copy-On-Write memory, so depending on what he's doing it might not take up as much memory if it were being run on Linux (or other OS with COW). Another thing that maybe the OP should look at (if he hasn't already) is to run a proxy in front of the main mod_perl application. Even if the proxy is on the same machine it will help because you can reduce the number of memory-heavy mod_perl processes/threads and handle the same number of connections. But, even after all that I have applications where we consistently run 3-4G just for mod_perl/Apache. -- Michael Peters Plus Three, LP
Re: experiencing Out of memory errors
Michael Peters schrieb am 27.01.2011 um 19:14 (-0500): > But, even after all that I have applications where we consistently > run 3-4G just for mod_perl/Apache. But surely not in one process as the OP said he'd like to do? -- Michael Ludwig
Re: experiencing Out of memory errors
On 01/27/2011 07:41 PM, Michael Ludwig wrote: Michael Peters schrieb am 27.01.2011 um 19:14 (-0500): But, even after all that I have applications where we consistently run 3-4G just for mod_perl/Apache. But surely not in one process as the OP said he'd like to do? No you're right, but I'm guessing he might be running a threaded MPM, so single process, multiple threads. -- Michael Peters Plus Three, LP
Re: experiencing Out of memory errors
On 1/27/2011 7:16 PM, Michael Peters wrote: > On 01/27/2011 07:41 PM, Michael Ludwig wrote: >> Michael Peters schrieb am 27.01.2011 um 19:14 (-0500): >> >>> But, even after all that I have applications where we consistently >>> run 3-4G just for mod_perl/Apache. >> >> But surely not in one process as the OP said he'd like to do? > > No you're right, but I'm guessing he might be running a threaded MPM, so > single process, > multiple threads. Exactly. Even constraining httpd to smaller stacks is unlikely to be wise with mod_perl running, consider the default is 256kb IIRC. Shrinking this to 128k obviously is a big help, but is unrealistic. I agree with Peters, run a proxy in front of the server hosting mod_perl. Even when we ship binary 64 bit for win32, there are far too many broken perl and similar modules which don't expect sizeof(long*) > sizeof(long), which has tripped up many porters. Most of these defects are gone from apr and httpd, and perl and modperl can likely catch up quickly, but to isolate every possible modperl XS package on cpan and identify all those which make stupid long x = (long)xptr; assignments will be arduous.