RE: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
Memory management is tricky though. Hard to tell which values sum up to
the
real thing.
Probably best meter on Linux is the actual free value highlighted
below?
Check it before starting amavisd/spamd/whatnot and check it again after
running for a while. Also double check it after killing all the
processes.
I'm open to be proved otherwise..
$ free
total used free sharedbuffers
cached
Mem: 1047496 944236 103260 0 2904
284336
-/+ buffers/cache: 656996 ___390500___
Swap: 524272 28 257604
Let's see if Private_ entries in smaps do the right thing.
This is the command I use to get allocated private memory from a process (in
kB):
awk 'BEGIN {p=0;} $1 ~ /Private_/ {p += $2;} END {print p;}'
/proc/PID/smaps
Besides, one can specify the smaps file of more than a process, not only
one.
Regards,
Giampaolo
Re: fork is vfork?
On Don, 2013-03-07 at 13:12 -0500, David F. Skoll wrote: On Thu, 07 Mar 2013 19:04:22 +0100 Bernd Petrovitsch be...@petrovitsch.priv.at wrote: MD forks the worker process and the worker process initializes libperl and loads the perl script. Nope. Hmm, I had the impression ages ago with md-2.56 - did this change since then? [ And I mean with embedded-perl compiled-in and activated. ] To share more memory on a fork, it should initialize libperl and load the perl script before forking off the worker processes. That's what it does. Then I have no idea, why the mimedefang-multiplexor processes have next to nothing shared - except that perl actually changes a lot in the internal data structures killing any copy-on-write benefits. Hmm, perl probably does not separate code from data internally that much . Bernd -- Bernd Petrovitsch Email : be...@petrovitsch.priv.at LUGA : http://www.luga.at
Re: fork is vfork?
On Fri, 08 Mar 2013 13:26:50 +0100 Bernd Petrovitsch be...@petrovitsch.priv.at wrote: Hmm, I had the impression ages ago with md-2.56 - did this change since then? [ And I mean with embedded-perl compiled-in and activated. ] No, I think it's been that way for ages. Then I have no idea, why the mimedefang-multiplexor processes have next to nothing shared - except that perl actually changes a lot in the internal data structures killing any copy-on-write benefits. I think that's what I said originally... but was met with skepticism from some. Regards, David.
Re: fork is vfork?
On Fri, Mar 08, 2013 at 09:09:27AM -0500, David F. Skoll wrote:
I think that's what I said originally... but was met with skepticism from
some.
There is a difference in saying something and actually providing some data.
I'm sorry but this sounds like True Believers (no need to prove anything)
vs Scepticists (no matter what you prove, it doesn't matter to them). :-)
Here is my full documentation. I don't really care if you are on sa-users
list and claim to not even have access to spamd, but if we keep going, let's
base the claims on hard data? If MD behaves differently, that's another
matter and not relevant to SA.
$ uname -a
Linux ubuntu 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64
x86_64 x86_64 GNU/Linux
$ /usr/local/perl/bin/perl -v
This is perl 5, version 16, subversion 2 (v5.16.2) built for x86_64-linux
$ free
total used free sharedbuffers cached
Mem: 40495843593572 456012 0 2903801636144
-/+ buffers/cache:16670482382536
Swap:0 0 0
(even disabled swap so it doesn't interfere)
$ /usr/local/perl/bin/spamd -4 -p 1234 -m 50 --min-children=50 --min-spare=40
--max-conn-per-child=1000 --round-robin -L -d
$ pgrep -f spamd |wc -l
51
$ free
total used free sharedbuffers cached
Mem: 40495843730644 318940 0 2903961636152
-/+ buffers/cache:18040962245488
Swap:0 0 0
(memory difference to before spamd running: 137MB)
$ pgrep -f 'spamd child' | while read p; do grep Private_ /proc/$p/smaps; done
| awk 'BEGIN {p=0;} $1 ~ /Private_/ {p += $2;} END {print p;}'
78700
(smaps claims virgin childs are using 78MB / 50 = ~1.5MB/child)
$ find ham.old -type f | while read f; do spamc -p 1234 $f; done
(zzz. about 25 messages processed per child)
$ ps axu |grep spamd
hege 23755 0.2 1.0 121608 44244 ?Ss 17:23 0:01
/usr/local/perl/bin/spamd -4 -p 1234 -m 50 --min-children=50 --min-spare=40
--max-conn-per-child=1000 --round-robin -L -d
hege 23756 2.1 1.2 129400 52400 ?S17:23 0:09 spamd child
hege 23757 2.0 1.3 130432 53284 ?S17:23 0:09 spamd child
hege 23758 1.0 1.2 127168 50244 ?S17:23 0:04 spamd child
hege 23759 0.9 1.2 129636 52628 ?S17:23 0:04 spamd child
hege 23760 1.0 1.2 127984 50960 ?S17:23 0:04 spamd child
hege 23761 1.2 1.3 130204 53080 ?S17:23 0:05 spamd child
hege 23762 1.1 1.2 128312 51324 ?S17:23 0:05 spamd child
hege 23763 1.4 1.2 126356 49512 ?S17:23 0:06 spamd child
hege 23764 0.9 1.2 128636 51408 ?S17:23 0:04 spamd child
hege 23765 1.2 1.2 129112 52056 ?S17:23 0:05 spamd child
hege 23766 1.1 1.2 128048 51068 ?S17:23 0:05 spamd child
hege 23767 1.0 1.2 127604 50588 ?S17:23 0:04 spamd child
hege 23768 1.2 1.2 129244 52376 ?S17:23 0:05 spamd child
hege 23769 0.9 1.2 128680 51632 ?S17:23 0:04 spamd child
hege 23770 1.8 1.3 133168 56184 ?S17:23 0:08 spamd child
hege 23771 0.8 1.2 129112 52096 ?S17:23 0:03 spamd child
hege 23772 0.6 1.2 128268 51220 ?S17:23 0:03 spamd child
hege 23773 2.4 1.3 132232 55132 ?S17:23 0:11 spamd child
hege 23774 1.3 1.2 129260 52288 ?S17:23 0:06 spamd child
hege 23775 1.0 1.3 130308 53360 ?S17:23 0:04 spamd child
hege 23776 1.8 1.2 126960 49876 ?S17:23 0:08 spamd child
hege 23777 1.0 1.2 128608 51692 ?S17:23 0:04 spamd child
hege 23778 0.9 1.2 127232 50248 ?S17:23 0:04 spamd child
hege 23779 1.5 1.2 128808 51936 ?S17:23 0:07 spamd child
hege 23780 1.1 1.3 131680 54504 ?S17:23 0:05 spamd child
hege 23781 1.5 1.2 126596 49760 ?S17:23 0:07 spamd child
hege 23782 1.0 1.3 130636 53632 ?S17:23 0:04 spamd child
hege 23783 1.0 1.3 130816 53716 ?S17:23 0:04 spamd child
hege 23784 1.7 1.3 132872 55820 ?S17:23 0:08 spamd child
hege 23785 0.8 1.3 131696 54608 ?S17:23 0:03 spamd child
hege 23786 2.1 1.3 130104 53208 ?S17:23 0:10 spamd child
hege 23787 1.1 1.3 130940 53892 ?S17:23 0:05 spamd child
hege 23788 1.0 1.3 130116 52932 ?S17:23 0:04 spamd child
hege 23789 1.3 1.3 132556 55388 ?S17:23 0:06 spamd child
hege 23790 0.8 1.2 127744 50784 ?S17:23 0:04 spamd child
hege 23791 1.1 1.2 125812 48984 ?S17:23 0:05 spamd child
hege 23792 1.6 1.3 129812 52804 ?S17:23 0:07 spamd child
hege 23793 0.8 1.3
Re: fork is vfork?
On Fri, 8 Mar 2013 17:42:58 +0200
Henrik K h...@hege.li wrote:
$ pgrep -f 'spamd child' | while read p; do grep Private_ /proc/$p/smaps;
done | awk 'BEGIN {p=0;} $1 ~ /Private_/ {p += $2;} END {print p;}'
I would be interested in seeing the output of:
pgrep -f 'spamd child' | while read p; do grep Shared_ /proc/$p/smaps; done |
awk 'BEGIN {p=0;} $1 ~ /Shared_/ {p += $2;} END {print p;}'
Regards,
David.
Re: fork is vfork?
On Fri, Mar 08, 2013 at 11:26:39AM -0500, David F. Skoll wrote:
On Fri, 8 Mar 2013 17:42:58 +0200
Henrik K h...@hege.li wrote:
$ pgrep -f 'spamd child' | while read p; do grep Private_ /proc/$p/smaps;
done | awk 'BEGIN {p=0;} $1 ~ /Private_/ {p += $2;} END {print p;}'
I would be interested in seeing the output of:
pgrep -f 'spamd child' | while read p; do grep Shared_ /proc/$p/smaps; done |
awk 'BEGIN {p=0;} $1 ~ /Shared_/ {p += $2;} END {print p;}'
Here's a new run..
Virgin childs:
$ pgrep -f 'spamd child' | while read p; do grep Private_ /proc/$p/smaps; done
| awk 'BEGIN {p=0;} $1 ~ /Private_/ {p += $2;} END {print p;}'
79364
$ pgrep -f 'spamd child' | while read p; do grep Shared_ /proc/$p/smaps; done |
awk 'BEGIN {p=0;} $1 ~ /Shared_/ {p += $2;} END {print p;}'
2057688
Used childs:
$ pgrep -f 'spamd child' | while read p; do grep Private_ /proc/$p/smaps; done
| awk 'BEGIN {p=0;} $1 ~ /Private_/ {p += $2;} END {print p;}'
1348048
$ pgrep -f 'spamd child' | while read p; do grep Shared_ /proc/$p/smaps; done |
awk 'BEGIN {p=0;} $1 ~ /Shared_/ {p += $2;} END {print p;}'
1271252
Re: fork is vfork?
On Fri, 8 Mar 2013 18:44:54 +0200 Henrik K h...@hege.li wrote: Virgin childs: 79MB private; 2GB shared (~40MB shared/child) Used childs: 1.2GB private; 1.2GB shared (~24MB shared/child) This is roughly what I am seeing with MIMEDefang also: Only 50% shared. It's better than nothing, but not nearly as good as one might have hoped. On my system, about 3.5MB of the shared memory is the text portion of libraries such as libperl.so and libc.so which you'd expect to be shared anyway, so really only about 20.5MB/46.5MB of Perl memory is shared for each slave. Regards, David.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 14:18:12 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: I'm not talking about the semantics but about the implementation. Simply said, vfork() was developed to avoid process memory copying used at fork(). on linux, fork() does NOT copy process memory. On 07.03.13 09:48, David F. Skoll wrote: vfork() also suspends execution of the parent until the child calls execve or _exit. If the child happens to write into its memory, the parent sees the changes... very different from fork(). I think Giampaolo Tomassoni got the point in his reply to the same mail I was replying to. Now, as for the great benefits of copy-on-write: It is actually almost useless with Perl programs. Here's the reason: Perl uses reference-counting to know when to free memory. So even if you access memory read-only by creating a new reference to the underlying object, that effectively becomes a write operation and Linux needs to copy the page. luckily, this does not happen at fork() time but at the time memory is changed. Mamory may stay unchanged, so even after some time the memory footprint can be smaller. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. One World. One Web. One Program. - Microsoft promotional advertisement Ein Volk, ein Reich, ein Fuhrer! - Adolf Hitler
Re: fork is vfork?
On Fri, Mar 08, 2013 at 11:55:43AM -0500, David F. Skoll wrote: On Fri, 8 Mar 2013 18:44:54 +0200 Henrik K h...@hege.li wrote: Virgin childs: 79MB private; 2GB shared (~40MB shared/child) Used childs: 1.2GB private; 1.2GB shared (~24MB shared/child) This is roughly what I am seeing with MIMEDefang also: Only 50% shared. It's better than nothing, but not nearly as good as one might have hoped. On my system, about 3.5MB of the shared memory is the text portion of libraries such as libperl.so and libc.so which you'd expect to be shared anyway, so really only about 20.5MB/46.5MB of Perl memory is shared for each slave. But atleast ~10-20MB of private data per spamd child is the per message scan data/blobs etc (can be seen as 20mb heap allocation). This should not be calculated in any memory ratio conclusions.
Re: fork is vfork?
On Fri, Mar 08, 2013 at 07:09:12PM +0200, Henrik K wrote: On Fri, Mar 08, 2013 at 11:55:43AM -0500, David F. Skoll wrote: On Fri, 8 Mar 2013 18:44:54 +0200 Henrik K h...@hege.li wrote: Virgin childs: 79MB private; 2GB shared (~40MB shared/child) Used childs: 1.2GB private; 1.2GB shared (~24MB shared/child) This is roughly what I am seeing with MIMEDefang also: Only 50% shared. It's better than nothing, but not nearly as good as one might have hoped. On my system, about 3.5MB of the shared memory is the text portion of libraries such as libperl.so and libc.so which you'd expect to be shared anyway, so really only about 20.5MB/46.5MB of Perl memory is shared for each slave. But atleast ~10-20MB of private data per spamd child is the per message scan data/blobs etc (can be seen as 20mb heap allocation). This should not be calculated in any memory ratio conclusions. Also to be noted that these shared values are completely vague anyway. All that matters is how much real system memory is taken. As seen by my previous free reports, master + 50 virgin childs only take total of 137MB. At full blast everything take ~1.2GB. So the real ratio per child might be something like 4MB bogus perl data, 20MB of per-message data. If the master process takes 50MB memory, this means the child ratio is _much_ better than 50% (4MB/50MB). I might dig deeper into this later with some tools.
fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 08:57:28 +0100 Giampaolo Tomassoni giampa...@tomassoni.biz wrote: I don't see too many differences with running more SA processes with linuxes (in which a fork() is actually a vfork()). I don't believe that's true. Do you have evidence to back up that claim? fork() and vfork() have very different semantics and vfork() would not work at all for spamd. Regards, David.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 08:57:28 +0100 Giampaolo Tomassoni giampa...@tomassoni.biz wrote: I don't see too many differences with running more SA processes with linuxes (in which a fork() is actually a vfork()). On 07.03.13 07:01, David F. Skoll wrote: I don't believe that's true. Do you have evidence to back up that claim? fork() and vfork() have very different semantics and vfork() would not work at all for spamd. the implementation of fork() in linux makes it nearly the same as vfork(). -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. WinError #9: Out of error messages.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 13:47:55 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: the implementation of fork() in linux makes it nearly the same as vfork(). That is completely wrong. Just because modern forks use copy-on-write doesn't make them anything at all like vfork; the semantics are utterly different. Regards, David.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 13:47:55 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: the implementation of fork() in linux makes it nearly the same as vfork(). On 07.03.13 07:53, David F. Skoll wrote: That is completely wrong. Just because modern forks use copy-on-write doesn't make them anything at all like vfork; the semantics are utterly different. I'm not talking about the semantics but about the implementation. Simply said, vfork() was developed to avoid process memory copying used at fork(). on linux, fork() does NOT copy process memory. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. WinError #98652: Operation completed successfully.
RE: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 13:47:55 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: the implementation of fork() in linux makes it nearly the same as vfork(). That is completely wrong. Just because modern forks use copy-on-write doesn't make them anything at all like vfork; the semantics are utterly different. Uhu! You need to put things in their own context in order to get the semantic. Should I had to say: a fork under linux attains performances as close to a vfork? I'm replying to a list, not writing a CS book, come on... Giampaolo Regards, David.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 14:18:12 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: I'm not talking about the semantics but about the implementation. Simply said, vfork() was developed to avoid process memory copying used at fork(). on linux, fork() does NOT copy process memory. vfork() also suspends execution of the parent until the child calls execve or _exit. If the child happens to write into its memory, the parent sees the changes... very different from fork(). Now, as for the great benefits of copy-on-write: It is actually almost useless with Perl programs. Here's the reason: Perl uses reference-counting to know when to free memory. So even if you access memory read-only by creating a new reference to the underlying object, that effectively becomes a write operation and Linux needs to copy the page. I think if you measure what happens to Perl processes that fork a number of children to handle requests, you'll see that there's very little memory sharing after a short while. Regards, David.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, Mar 07, 2013 at 09:48:19AM -0500, David F. Skoll wrote: I think if you measure what happens to Perl processes that fork a number of children to handle requests, you'll see that there's very little memory sharing after a short while. Please let's stop the techno-theorizing and provide actual results. We already had this exact same discussion atleast once *sigh*. Start something like: spamd -4 -p 1234 --min-children=50 --min-spare=50 --max-conn-per-child=1000 --round-robin -L 50 non-recycled childs, fed 1000 requests (~20 each). Memory measured with free (without buffers/cache etc): begin 2588084 end 1296756 About 25MB non-shared memory used per child, which is pretty normal since SA uses lots of internal per-message data. On 32-bit systems the usage could be half of that. So in the case of SA, it's not anywhere near very little memory shared after a short while.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 17:47:22 +0200 Henrik K h...@hege.li wrote: Memory measured with free (without buffers/cache etc): begin 2588084 end 1296756 About 25MB non-shared memory used per child, Are you sure your measurements are correct? I use MIMEDefang which also has a preforked-children architecture and I see only about 4MB shared per child with the vast majority of per-child memory non-shared. This is based on what top reports. So in the case of SA, it's not anywhere near very little memory shared after a short while. My measurements completely disagree with yours, so one of us (or both?) is wrong. Regards, David.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, Mar 07, 2013 at 11:37:33AM -0500, David F. Skoll wrote: On Thu, 7 Mar 2013 17:47:22 +0200 Henrik K h...@hege.li wrote: Memory measured with free (without buffers/cache etc): begin 2588084 end 1296756 About 25MB non-shared memory used per child, Are you sure your measurements are correct? I use MIMEDefang which also has a preforked-children architecture and I see only about 4MB shared per child with the vast majority of per-child memory non-shared. This is based on what top reports. You provide no data how you end up with the 4MB etc. And MD is not SA, it might do all sorts of funky stuff. How about actually trying the provided spamd line yourself and not keep again theorizing how someone is measuring wrong etc? Well actually here is the one I used to get 50 childs.. pasted wrong one. spamd -4 -p 1234 -m 50 --min-children=50 --min-spare=40 --max-conn-per-child=1000 --round-robin -L Just feed a lot of random messages with spamc -p 1234.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, 7 Mar 2013 18:56:45 +0200 Henrik K h...@hege.li wrote: You provide no data how you end up with the 4MB etc. And MD is not SA, it might do all sorts of funky stuff. I wrote MD, so I'm pretty sure it's not doing any funky stuff. How about actually trying the provided spamd line yourself and not keep again theorizing how someone is measuring wrong etc? I don't have any machine with spamd installed. Regards, David.
RE: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, Mar 07, 2013 at 11:37:33AM -0500, David F. Skoll wrote:
On Thu, 7 Mar 2013 17:47:22 +0200
Henrik K h...@hege.li wrote:
Memory measured with free (without buffers/cache etc):
begin 2588084
end 1296756
About 25MB non-shared memory used per child,
Are you sure your measurements are correct? I use MIMEDefang which
also
has a preforked-children architecture and I see only about 4MB shared
per child with the vast majority of per-child memory non-shared.
This
is based on what top reports.
You provide no data how you end up with the 4MB etc. And MD is not SA,
it
might do all sorts of funky stuff.
How about actually trying the provided spamd line yourself and not keep
again theorizing how someone is measuring wrong etc?
Well actually here is the one I used to get 50 childs.. pasted wrong
one.
spamd -4 -p 1234 -m 50 --min-children=50 --min-spare=40 --max-conn-per-
child=1000 --round-robin -L
Just feed a lot of random messages with spamc -p 1234.
I just got a snip into my amavisd's 5 children /proc/pid/smaps file,
summing together the count of Private_{Clean|Dirty} pages.
I got this:
p1: 74,164 kb
p2: 70,772 kb
p3: 71,548 kb
p4: 74,064 kb
p5: 70,784 kb
This accounts for a total of unique 287,168 kB (say 280 MB?). ~ 56MB in the
average.
Sounds this good?
Giampaolo
Re: fork is vfork?
On Don, 2013-03-07 at 12:14 -0500, David F. Skoll wrote: On Thu, 7 Mar 2013 18:56:45 +0200 Henrik K h...@hege.li wrote: You provide no data how you end up with the 4MB etc. And MD is not SA, it might do all sorts of funky stuff. I wrote MD, so I'm pretty sure it's not doing any funky stuff. MD forks the worker process and the worker process initializes libperl and loads the perl script. To share more memory on a fork, it should initialize libperl and load the perl script before forking off the worker processes. BTW switching to vfork() won't speed up anything significantly IMHO as processes are (re)started quite seldom - it is not that one starts a process for each mail . Bernd -- Bernd Petrovitsch Email : be...@petrovitsch.priv.at LUGA : http://www.luga.at
Re: fork is vfork?
On Thu, 07 Mar 2013 19:04:22 +0100 Bernd Petrovitsch be...@petrovitsch.priv.at wrote: MD forks the worker process and the worker process initializes libperl and loads the perl script. Nope. To share more memory on a fork, it should initialize libperl and load the perl script before forking off the worker processes. That's what it does. Regards, DAvid.
Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
On Thu, Mar 07, 2013 at 07:02:00PM +0100, Giampaolo Tomassoni wrote:
I just got a snip into my amavisd's 5 children /proc/pid/smaps file,
summing together the count of Private_{Clean|Dirty} pages.
I got this:
p1: 74,164 kb
p2: 70,772 kb
p3: 71,548 kb
p4: 74,064 kb
p5: 70,784 kb
This accounts for a total of unique 287,168 kB (say 280 MB?). ~ 56MB in the
average.
Sounds this good?
Memory management is tricky though. Hard to tell which values sum up to the
real thing.
Probably best meter on Linux is the actual free value highlighted below?
Check it before starting amavisd/spamd/whatnot and check it again after
running for a while. Also double check it after killing all the processes.
I'm open to be proved otherwise..
$ free
total used free sharedbuffers cached
Mem: 1047496 944236 103260 0 2904 284336
-/+ buffers/cache: 656996 ___390500___
Swap: 524272 28 257604
R: Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)
The Private_ entries in /proc/.../smaps are reported to be the right choice
here: they report only pages allocated while not shared with any other process.
Ie, the ones touched after fork and the new allocated ones.
Also, smaps is a relatively new proc entry, meant exactly to cope with all the
linux memory stats mess.
Giampaolo
Henrik K h...@hege.li ha scritto:
On Thu, Mar 07, 2013 at 07:02:00PM +0100, Giampaolo Tomassoni wrote:
I just got a snip into my amavisd's 5 children /proc/pid/smaps file,
summing together the count of Private_{Clean|Dirty} pages.
I got this:
p1: 74,164 kb
p2: 70,772 kb
p3: 71,548 kb
p4: 74,064 kb
p5: 70,784 kb
This accounts for a total of unique 287,168 kB (say 280 MB?). ~ 56MB in the
average.
Sounds this good?
Memory management is tricky though. Hard to tell which values sum up to the
real thing.
Probably best meter on Linux is the actual free value highlighted below?
Check it before starting amavisd/spamd/whatnot and check it again after
running for a while. Also double check it after killing all the processes.
I'm open to be proved otherwise..
$ free
total used free shared buffers cached
Mem: 1047496 944236 103260 0 2904 284336
-/+ buffers/cache: 656996 ___390500___
Swap: 524272 28 257604
