* On 2004.09.15, in <[EMAIL PROTECTED]>,
* "Fajar A. Nugraha" <[EMAIL PROTECTED]> wrote:
>
> Do you have any suggestion as to how to get back the free()d memory?
> Will (borrowing Apache's way) using a prefork-kind of daemon, with
> limited lifetime
> for each child, be better (in sense of memory management) than the current
> thread implementation? Or perhaps limiting the lifetime of each thread
> sufficient?
I wouldn't say better, necessarily. Using processes instead of
lightweight processes (LWPs, or "threads") for your threads of execution
will certainly return memory to the system more often. It will also
consume more memory per thread of execution -- realize that all the
BSS and heap data for a single process will be reduplicated for each
process, and that you'll have a process for each concurrent e-mail
message being scanned. Currently a lot of that is shared once among
all concurrent messages. It will also force the kernel to spend more
time mapping and unmapping memory pages, and you won't get the same I/O
cache benefit. It might retain less memory at some sample points (though
not at others), but I can't say that it will make your system perform
better, which is the real issue, I suspect.
Remember: ps tells you that some amount of memory is mapped to the
process. It does not tell you what that memory is being used for. If
that's all used to accelerate disk I/O, that seems good, not bad.
The problem mentioned in the subject was someone else's issue, running
clamd on Red Hat 9, in conjunction MIMEdefang. I don't know much about
that combination, and can't speak to it. I'm not sure what problem
you're seeing -- but big numbers next to "clamd" in ps output isn't a
problem, per se, and putting monitors and such around clamd to make it
terminate early can hurt your overall performance. (Sorry if I've missed
details to your situation -- I don't usually get around to everything on
this list.)
Certainly, I think that in general it benefits you to use multi-threaded
(MT) code, and to retain all the memory you can in the MT process,
*without* returning it to the system. At small e-mail volumes this
doesn't matter much, but as your volume increases, this matters more,
and more rapidly. A kernel can spend a lot of time spawning processes
and remapping memory, but MT code and memory conservation are strategies
to minimize this.
If you're experiencing software failures or memory shortages in other
applications on the server, you need to look at general performance
tuning for the whole system, and maybe break services out onto multiple
systems. Performance tuning is a holistic process -- there are a lot of
interrelated things to take into account, and you might well need to
broaden your toolset. Sar will help, as will a variety of analysis tools
that work on sar data.
There are good books on the topic, too. Adrian Cockroft, Richard
McDougall, and Jim Mauro all have excellent materials on the topic,
besides having been instrumental in the Solaris VM architecture. Among
other resources, they refer their students to:
http://www.solarisinternals.com/
http://www.sun.com/blueprints/
http://www.sun.com/bigadmin/
--
-D. [EMAIL PROTECTED] NSIT::ENSS
-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM.
Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php
_______________________________________________
Clamav-users mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/clamav-users
