I hate to just 'me too' on these discussions, but I agree. In both 3.4.2 and 4.5 (the only two AOLServers we have any experience with here), our typical experience has been that leaks were in our application area, and many of the most nasty ones to find were in long-lived threads. We've tried to find them using Purify, valgrind, and a stack-trace we built into the Windows memory allocator for 3.4.2; of these tools, valgrind has been the most useful.
Just to add something to the discussion: One point I think is confusing about the C API, and needs more amplification in the documentation, is the asymmetry between Ns_TclAllocateInterp and Nn_TclDeAllocateInterp. In our C/C++ modules, we often need to "get the current interp and run this Tcl command", which may cause recursion back into our module(s). And our module can be used from either conn threads or non-conn (timer, multicast reception, etc) threads. We don't want to leak Interps, but we also don't want to leak ns_sets and other interp-specific Tcl contexts that should be cleaned up between requests. We tried to do this by only allocating the Interp on demand, but then the question of when to deallocate was very confusing. You don't want to just Alloc at the beginning of a block, and then DeAlloc at the end, the way a C++ programmer would usually think. In fact, that usually breaks things, because the Alloc didn't actually Alloc anything, but the DeAlloc cleans everything up, so ns_sets and Tcl globals that were perfectly fine before you called into our module would just "disappear". Instead, you have to grok the AOLServer driver/conn model, and have your timer/receiver/whatever thread somehow know whether the demand was made, and then deallocate only as the complete request context goes out of scope. So: I hate the names of those functions. Symmetric names should imply symmetric operations. In the end, we constructed our own "KnContext" stack (in C++) we could use symmetrically. To help ourselves out with performance monitoring, we instrumented that so we could see leaks of interps, and also count number of allocated sets (well, last set# allocated) and response time thresholds as we unwound them. (Of course, we put all that in our own modules, not the core AOLServer, so we don't get those benefits automatically when using adps or registered Tcl procs; there's probably some way to pull that off with traces, but we were never motivated to try.) -- ReC -----Original Message----- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Maurizio Martignano Sent: Friday, May 02, 2008 1:48 AM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Getting a handle on memory usage Dear all, If a proper analysis of the memory consumption is of interest to anyone, it could be an idea to use Valgrind/Memcheck: http://valgrind.org/ Anyhow, before going over such detailed and time consuming analysis, I first would try to: 1. use the TCL Standard memory allocator (and not the Zippy one) 2. properly configure the parameters in the 'config.tcl' Cheers, Maurizio -----Original Message----- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Dossy Shiobara Sent: 01 May 2008 21:11 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Getting a handle on memory usage On 2008.05.01, Maurizio Martignano <[EMAIL PROTECTED]> wrote: > When using Zippy I noticed that the memory occupied by Aolserver (nsd) > always grows, without being released. That's probably because we don't invoke madvise() anywhere. Of course, apparently on MacOS X madvise() is broken, and on Win32 we should use VirtualFree(). See this entry about jemalloc where this issue is explored: Perceived jemalloc memory footprint http://www.canonware.com/~ttt/2008/01/perceived-jemalloc-memory-footprin t.ht ml I mentioned on the IRC chat that it might be fun to implement a very fine-grained debugging memory allocator implementation that keeps lots of statistics that we can examine and is designed with knowledge that it's debugging an AOLserver process to record the relevant bits of information so we can correlate it back to an activity that the code is performing. I don't know if this is a worthwhile idea, but it was fun to think about for a moment, anyway. -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
