Eric Frazier wrote:
Hi,
This may be totaly ignorate crap, but I noticed this when I was reading the
ps man page on BSD 4.5 about sys/proc.h flags
This one I noticed..
P_SYSTEM 0x00200System proc: no sigs, stats or swapping
Could this mean what I think it means? That
Hi all,
On Sat, 16 Mar 2002, Bill Marrs wrote:
leads ones to wonder if some of our assumptions or tools used to
monitor memory are inaccurate or we're misinterpreting them.
Well 'top' on Linux is rubbish for sure.
73,
Ged.
Hi,
I had hoped that FreeBSD would be immune, but it seems not. I have been
bashing it with http_load and all of a sudden(after a LOT of bashing and
swaping) all of my processes had zero shared. It did take me days of
fiddling to run into this though.
Thanks,
Eric
At 04:16 PM 3/16/02
Hi,
This may be totaly ignorate crap, but I noticed this when I was reading the
ps man page on BSD 4.5 about sys/proc.h flags
This one I noticed..
P_SYSTEM 0x00200System proc: no sigs, stats or swapping
Could this mean what I think it means? That a process with this flag set,
I believe I have the answer...
The problem is that the parent httpd swaps, and any new children it
creates load the portion of memory that was swaped from swap, which does
not make it copy-on-write. The really annoying thing - when memory gets
tight, the parent is the most likely httpd process
Yes, this is my theory also. I figured this out a while back, and started a
thread on this list, but since then haven't had enough time to investigate
it further.
The thread is here:
http://mathforum.org/epigone/modperl/wherdtharvoi
which includes some helpful hints from Doug on how to call
The reason turning off swap works is because it forces the memory from
the parent process that was swapped out to be swapped back in. It will
not fix those processes that have been sired after the shared memory
loss, as of Linux 2.2.15 and Solaris 2.6. (I have not checked since
then for
It's copy-on-write. The swap is a write-to-disk.
There's no such thing as sharing memory between one process on disk(/swap)
and another in memory.
agreed. What's interesting is that if I turn swap off and back on again,
the sharing is restored! So, now I'm tempted to run a crontab every 30
On Thu, 14 Mar 2002 07:25:27 -0500, Bill Marrs [EMAIL PROTECTED] said:
It's copy-on-write. The swap is a write-to-disk.
There's no such thing as sharing memory between one process on disk(/swap)
and another in memory.
agreed. What's interesting is that if I turn swap off and back
Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Thursday, March 14, 2002 8:24 AM
To: Bill Marrs
Cc: [EMAIL PROTECTED]
Subject: Re: loss of shared memory in parent httpd
On Thu, 14 Mar 2002 07:25:27 -0500, Bill Marrs [EMAIL PROTECTED]
said:
It's copy-on-write. The swap
How is it even remotely possible that turning off swap restores memory
shared between processes? Is the Linux kernel going from process to process
comparing pages of memory as they re-enter RAM? Oh, those two look
identical, they'll get shared?
This is a good point. I really have no clue how
On Thu, 14 Mar 2002, Bill Marrs wrote:
It's copy-on-write. The swap is a write-to-disk.
There's no such thing as sharing memory between one process on disk(/swap)
and another in memory.
agreed. What's interesting is that if I turn swap off and back on again,
what? doesn't seem to me
Bill Marrs wrote:
You actually can do this. See the mergemem project:
http://www.complang.tuwien.ac.at/ulrich/mergemem/
I'm interested in this, but it involves a kernel hack and the latest
version is from 29-Jan-1999, so I got cold feet.
It was a student project. And unless someone
I just wanted to mention that the theory that my loss of shared memory in
the parent is related to swapping seems to be correct.
When the lack of sharing occurs, it is correlated with my httpd processes
showing a SWAP (from top/ps) of 7.5MB, which is roughly equal to the amount
of sharing
of shared memory in parent httpd
On Tue, 12 Mar 2002, Graham TerMarsch wrote:
[...]
We saw something similar here, running on Linux servers. Turned out to be
that if the server swapped hard enough to swap an HTTPd out, then you
basically lost all the shared memory that you had. I can't
Stas Bekman wrote:
Bill Marrs wrote:
One more piece of advice: I find it easier to tune memory control
with a single parameter. Setting up a maximum size and a minumum
shared size is not as effective as setting up a maximum *UNSHARED*
size. After all, it's the amount of real memory
I'm a heavy mod_perl user, running 3 sites as virtual servers, all with
lots of custom Perl code. My httpd's are huge(~50mb), but with the help of
a startup file I'm able to get them sharing most of their
memory(~43). With the help of GTopLimit, I'm able to keep the memory usage
under
At 09:18 AM 3/12/02 -0500, Bill Marrs wrote:
If anyone has any ideas what might cause the httpd parent (and new
children) to lose a big chunk of shared memory between them, please let me
know.
I've seen this happen many times. One day it works fine, the next you're
in trouble. And in my
Oops. Premature sending...
I have two ideas that might help:
- reduce number of global variables used, less memory pollution by lexicals
- make sure that you have the most up-to-date (kernel) version of your
OS. Newer Linux kernels seem to be a lot savvy at handling shared memory
than older
Bill Marrs wrote:
I'm a heavy mod_perl user, running 3 sites as virtual servers, all with
lots of custom Perl code. My httpd's are huge(~50mb), but with the help
of a startup file I'm able to get them sharing most of their
memory(~43). With the help of GTopLimit, I'm able to keep the
On Tue, 12 Mar 2002 09:18:32 -0500
Bill Marrs [EMAIL PROTECTED] wrote:
But... recently, something happened, and things have changed. After some
random amount of time (1 to 40 minutes or so, under load), the parent httpd
suddenly loses about 7-10mb of share between it and any new child it
Elizabeth Mattijsen wrote:
At 09:18 AM 3/12/02 -0500, Bill Marrs wrote:
If anyone has any ideas what might cause the httpd parent (and new
children) to lose a big chunk of shared memory between them, please
let me know.
I've seen this happen many times. One day it works fine, the
At 11:46 PM 3/12/02 +0800, Stas Bekman wrote:
I'm not sure whether my assessment of the problem is correct. I would
welcome any comments on this.
Nope Elizabeth, your explanation is not so correct. ;)
Too bad... ;-(
Shared memory is not about sharing the pre-allocated memory pool (heap
At 12:43 AM 3/13/02 +0800, Stas Bekman wrote:
Doug has plans for a much improved opcode tree sharing for mod_perl 2.0,
the details are kept as a secret so far :)
Can't wait to see that!
This topic is covered (will be) in the upcoming mod_perl book, where
we include the following reference
Elizabeth Mattijsen wrote:
Since Perl is basically all data, you would need to find a way of
localizing all memory that is changing to as few memory chunks as
possible.
That certainly would help. However, I don't think you can do that in
any easy way. Perl doesn't try to keep compiled
On Tuesday 12 March 2002 06:18, you wrote:
I'm a heavy mod_perl user, running 3 sites as virtual servers, all with
lots of custom Perl code. My httpd's are huge(~50mb), but with the help
of a startup file I'm able to get them sharing most of their
memory(~43). With the help of GTopLimit,
Bill Marrs wrote:
But... recently, something happened, and things have changed. After
some random amount of time (1 to 40 minutes or so, under load), the
parent httpd suddenly loses about 7-10mb of share between it and any new
child it spawns.
One possible reason is that a perl memory
Thanks for all the great advice.
A number of you indicated that it's likely due to my apache processes being
partially swapped to disk. That seems likely to me. I haven't had a
chance to prove that point, but when it does it again and I'm around, I
plan to test it with free/top (top has a
On Tue, 12 Mar 2002, Graham TerMarsch wrote:
[...]
We saw something similar here, running on Linux servers. Turned out to be
that if the server swapped hard enough to swap an HTTPd out, then you
basically lost all the shared memory that you had. I can't explain all of
the technical
No, I can't explain the nitty gritty either. :-)
Someone should write up a summary of this thread and ask in a
technical linux place, or maybe ask Dean Gaudet.
I believe this is a linux/perl issue... stand alone daemons exhibit the
same behaviour... e.g. if you've got a parent PERL daemon
Bill Marrs wrote:
One more piece of advice: I find it easier to tune memory control with
a single parameter. Setting up a maximum size and a minumum shared
size is not as effective as setting up a maximum *UNSHARED* size.
After all, it's the amount of real memory being used by each
31 matches
Mail list logo
