Let me add my 2 cents as well as i have spent a lot of time trying to
deal with ever-growing nsd process.
Currently i have nsd process (Naviserver) with size about 500Mb running
without restart for 15 days.
Tcl compiled without Zippy, with standard malloc-family. I tried it with
vtmalloc as well, the difference is not big, it gets shrinked faster but
overall size-wise it is similar to just standard Linux malloc.
The only bug change i had to make to Naviserver is to intriduces timeout
and max number of jobs to execute to the Tcl job facility and to
scheduling thread. Conn threads are controlled by maxconn and etc, DB
handles have idle and config parameters when to close connections. the
only missing part was to let sched and tcljob cleanup their resources.
For example i use a lot ns_job and sch_schedXXX, that measn all Tcl
interps allocted by sched and job threads stay forever and grow
unconditionally.
But with limits and config parameters that tell those threads when to
exit work perfectly. Hope those changes may be useful for aolserver code
as well.
Maurizio Martignano wrote:
Dear all,
Sometime ago I made some tests on the memory leaks of
aolserver+tcs+openacs.
I'd like to share my two cents worth of what I believe I found, hoping it
might help.
Aolserver uses TCL, and TCL's got three different memory managers:
1. Standard: using the malloc, free, etc... primitives of the OS (not so
fast and not using a lot of memory)
2. Zippy: multithreaded primitives specially developed for TCL (very fast
but using plenty of memory - this is the default mechanism)
3. VTMalloc: another multithreaded implementation, developed by third
parties, and somehow in between Standard and Zippy for what concerns
efficiency and memory occupation.
When using Zippy I noticed that the memory occupied by Aolserver (nsd)
always grows, without being released.
On the contrary when using the Standard memory allocator every now and then
memory gets released when not used. The total memory occupied by Aolserver
keeps growing but at a slower pace that in the previous case.
How to enable TCL Standard memory allocator?
Just call the following configure command:
configure -enable-threads
Then edit the Makefile, look for AC_FLAGS and remove from it the
-DUSE_THREAD_ALLOC=1 define (i.e. delete the string "-DUSE_THREAD_ALLOC=1").
Compile everything, and see how it goes....
With this change in place, I've seen some saving on the memory occupied by
nsd (around 30%).
Another point to look at very carefully is the overall system configuration
(config.tcl):
Parameters like
maxconnections
maxdropped
maxthreads
minthreads
threadtimeout
and
maxidle
maxopen
connections 1
have quite an effect on the total occupied memory.
Here there's no general recipe. My suggestion would be try to use the
smallest possible values that still allow the system to work.
Hope it helps,
Maurizio
-----Original Message-----
From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of
Fenton, Brian
Sent: 01 May 2008 19:30
To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] Getting a handle on memory usage
Hi Titi,
I know Gustaf Neumann had a script to find "application level leaks". If he
doesn't respond I can post it.
Brian
-----Original Message-----
From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of
Titi Alailima
Sent: 01 May 2008 17:31
To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] Getting a handle on memory usage
This isn't a very helpful response. I'm not even asking about ongoing
memory growth and overhead, at least not yet. I want to know what are the
parts that we (or at least some of us) _do_ understand and how can we at
least calculate those things so that:
1. we even have a sense of how much memory is being eaten up by things we
don't know
2. we know at least roughly what sort of impact we can make by adjusting the
parts we do know
As for these test workloads, can someone enlighten me more as to what they
might look like and how they would be used? Depending on what is involved,
I or someone I know may be able to figure something out to get us started in
the right direction.
Titi Ala'ilima
Lead Architect
MedTouch LLC
1100 Massachusetts Avenue
Cambridge, MA 02138
617.621.8670 x309
-----Original Message-----
From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On
Behalf Of Dossy Shiobara
Sent: Wednesday, April 30, 2008 3:10 PM
To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] Getting a handle on memory usage
On 2008.04.30, Titi Alailima <[EMAIL PROTECTED]> wrote:
What are the best ways of figuring out how the memory usage in
AOLserver is broken down? [...]
There's lots of overheads all over the place. Each Tcl_Obj structure,
every Tcl_DString buffer ... the AOLserver driver thread and all its
structures used to manage state ...
All these little things should (in theory) be inconsequential compared
to the larger picture of the application's memory usage, but there's a
good chance there are a few lost pointer leaks, etc.
--
Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
