Re: best way to discover this process's current memory usage, cross-platform?

2005-12-01 Thread mickie . liu
RalfGB wrote: Alex Martelli schrieb: MrJean1 [EMAIL PROTECTED] wrote: This may work on MacOS X. An initial, simple test does yield credible values. Definitely looks promising, thanks for the pointer. However, I am not a MacOS X expert. It is unclear which field of the

Re: best way to discover this process's current memory usage, cross-platform?

2005-12-01 Thread Fredrik Lundh
[EMAIL PROTECTED] wrote: mallinfo is available on most UNIX-like systems(Linux, Solaris, QNX, etc.) and is also included in the dlmalloc library (which works on win32). There is a small C extension module at http://hathawaymix.org/Software/Sketches/ which should give access to mallinfo()

Re: best way to discover this process's current memory usage, cross-platform?

2005-12-01 Thread MrJean1
Did you try the function I posted on Nov 15? It returns the high water mark, like sbrk(0) and works for RH Linux (which is dlmalloc, AFAIK). /Jean Brouwers PS) Here is that code again (for RH Linux only!) size_t hiwm (void) { /* info.arena - number of bytes allocated * info.hblkhd -

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-17 Thread Serge Orlov
Jack Diederich wrote: Electric Fence[1] uses the LD_PRELOAD method. I've successfully used it to track down leaks in a python C extension. If you look at the setup.py in probstat[2] you'll see #libraries = [efence] # uncomment to use ElectricFence which is a holdover from developing.

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-16 Thread Paul Boddie
Neal Norwitz wrote: Valgrind actually runs on PPC (32 only?) and amd64, but I don't think that's the way to go for this problem. +1 for understatement of the week. Here's a really screwy thought that I think should be portable to all Unixes which have dynamic linking. LD_PRELOAD. Similar

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-16 Thread Nicola Larosa
On the subject of memory statistics, I'm surprised no-one has mentioned top in this thread (as far as I'm aware): I would have thought such statistics would have been available to top and presented by that program. Talking about top, this article may be useful: On measuring memory usage

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-16 Thread Jack Diederich
On Tue, Nov 15, 2005 at 10:10:42PM -0800, Neal Norwitz wrote: Alex Martelli wrote: matt [EMAIL PROTECTED] wrote: Perhaps you could extend Valgrind (http://www.valgrind.org) so it works with python C extensions? (x86 only) Alas, if it's x86 only I won't even look into the task

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-16 Thread Alex Martelli
Paul Boddie [EMAIL PROTECTED] wrote: Neal Norwitz wrote: Valgrind actually runs on PPC (32 only?) and amd64, but I don't think that's the way to go for this problem. +1 for understatement of the week. Here's a really screwy thought that I think should be portable to all Unixes which

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-16 Thread [EMAIL PROTECTED]
Neal Norwitz wrote: Here's a really screwy thought that I think should be portable to all Unixes which have dynamic linking. LD_PRELOAD. You can create your own version of malloc (and friends) and free. You intercept each call to malloc and free (by making use of LD_PRELOAD), keep track

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-16 Thread Alex Martelli
[EMAIL PROTECTED] [EMAIL PROTECTED] wrote: ... Neal Norwitz wrote: Here's a really screwy thought that I think should be portable to all Unixes which have dynamic linking. LD_PRELOAD. You can create your own version of malloc (and friends) and free. You intercept each call to

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread Alex Martelli
Neal Norwitz [EMAIL PROTECTED] wrote: Alex Martelli wrote: So, I thought I'd turn to the wisdom of crowds... how would YOU guys go about adding to your automated regression tests one that checks that a certain memory leak has not recurred, as cross-platform as feasible? In particular,

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread Alex Martelli
MrJean1 [EMAIL PROTECTED] wrote: My suggestion would also be to use sbrk() as it provides a high-water mark for the memory usage of the process. That's definitely what I would have used in the '70s -- nowadays, alas, it ain't that easy. Below is the function hiwm() I used on Linux (RedHat).

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread MrJean1
For some more details on Linux' mallinfo, see ftp://gee.cs.oswego.edu/pub/misc/malloc.h and maybe function mSTATs() in glibc/malloc/malloc.c (RedHat). /Jean Brouwers -- http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread MrJean1
For some more details on Linux' mallinfo, see ftp://gee.cs.oswego.edu/pub/misc/malloc.h and maybe function mSTATs() in glibc/malloc/malloc.c (RedHat). /Jean Brouwers -- http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread matt
Perhaps you could extend Valgrind (http://www.valgrind.org) so it works with python C extensions? (x86 only) matt -- http://mail.python.org/mailman/listinfo/python-list

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread Alex Martelli
matt [EMAIL PROTECTED] wrote: Perhaps you could extend Valgrind (http://www.valgrind.org) so it works with python C extensions? (x86 only) Alas, if it's x86 only I won't even look into the task (which does sound quite daunting as the way to solve the apparently-elementary question how much

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread MrJean1
This may work on MacOS X. An initial, simple test does yield credible values. However, I am not a MacOS X expert. It is unclear which field of the malloc_statistics_t struct to use and how malloc_zone_statistics with zone NULL accumulates the stats for all zones. /Jean Brouwers #if _MACOSX

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread Alex Martelli
MrJean1 [EMAIL PROTECTED] wrote: This may work on MacOS X. An initial, simple test does yield credible values. Definitely looks promising, thanks for the pointer. However, I am not a MacOS X expert. It is unclear which field of the malloc_statistics_t struct to use and how

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-15 Thread Neal Norwitz
Alex Martelli wrote: matt [EMAIL PROTECTED] wrote: Perhaps you could extend Valgrind (http://www.valgrind.org) so it works with python C extensions? (x86 only) Alas, if it's x86 only I won't even look into the task (which does sound quite daunting as the way to solve the

best way to discover this process's current memory usage, cross-platform?

2005-11-14 Thread Alex Martelli
Having fixed a memory leak (not the leak of a Python reference, some other stuff I wasn't properly freeing in certain cases) in a C-coded extension I maintain, I need a way to test that the leak is indeed fixed. Being in a hurry, I originally used a qd hack...: if sys.platform in ('linux2',

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-14 Thread Steven D'Aprano
Not sure if I should start a new thread or not, but since this is closely related, I'll just leave it as is. Alex Martelli wrote: Having fixed a memory leak (not the leak of a Python reference, some other stuff I wasn't properly freeing in certain cases) in a C-coded extension I maintain, I

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-14 Thread Alex Martelli
Steven D'Aprano [EMAIL PROTECTED] wrote: Not sure if I should start a new thread or not, but since this is closely related, I'll just leave it as is. Alex Martelli wrote: Having fixed a memory leak (not the leak of a Python reference, some other stuff I wasn't properly freeing in

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-14 Thread Neal Norwitz
Alex Martelli wrote: So, I thought I'd turn to the wisdom of crowds... how would YOU guys go about adding to your automated regression tests one that checks that a certain memory leak has not recurred, as cross-platform as feasible? In particular, how would you code _memsize()

Re: best way to discover this process's current memory usage, cross-platform?

2005-11-14 Thread MrJean1
My suggestion would also be to use sbrk() as it provides a high-water mark for the memory usage of the process. Below is the function hiwm() I used on Linux (RedHat). MacOS X and Unix versions are straigthforward. Not sure about Windows. /Jean Brouwers #if _LINUX #include malloc.h size_t