Stephen Kellett wrote: > In message <[EMAIL PROTECTED]>, > > Sverker Nilsson <[EMAIL PROTECTED]> writes >
[Note that actually it was Hermann Maier that wrote the following but as quoted, it may look like it was I that wrote it.] > >> i need to find out the memory usage of a specific function that i use in > >> my program. this function does some recursive calculations and i want my > >> program to display the amount of memory the function used to calculate a > >> specific value. > > Python Memory Validator. > > Run your program to completion. > Switch to the hotspots tab. > Search for your function. > All memory used in that function will be shown in the tree (with the > effective callstack) underneath that function node in the tree. > > http://www.softwareverify.com I can't try it out easily since it doesn't seem to support Linux. Also looking at the home page I couldn't find any description of what it was actually doing. The info links didn't work, they showed blank pages with my browser (Netscape 3.01.) So if you would like to explain here, it would be helpful. Especially, it isn't clear to me what this means (requoting): > All memory used in that function Since Python has a reference-counting allocator, and various kinds of objects use specialized allocation pools, it seems unclear and implementation defined what is meant with 'all memory used'. I could think of some different things this could mean. 1. The total memory allocated, regardless of whether it is released again. If it is to mean really all the memory, it should include memory allocated by special allocators such as the allocator for ints. In this case, the following example function def f(): x = 0 while x < 1000000: x += 1 would have allocated (at least) 1000000 objects, one for each iteration since int's are immutable and one is allocated each time x is assigned to (although each one is also deallocated.) So according to the meaning (1.), the 'total memory used' would be 1000000 * the size of ints. [ Though with some alternative interpreter implementation it might be an entirely different value such as 0 bytes since it may not need to allocate every integer in the heap. Would 'allocating' on the stack count? In a CPU register?] Is something like this what would be reported by your system? 2. The second alternative meaning would be the memory allocated in the function, MINUS the memory deallocated in the function. Is this what is reported by your system? If the case is according to the 2nd alternative, 'memory used in' f would be 0 bytes since all memory allocated is deallocated, and for a function like the following def g(): return range(1000) it would be the size allocated for the list of length 1000 + any int objects allocated to fill the list. Given the above function g, its memory usage could be tested using heapy and the following function. def test(g): from guppy import hpy hp=hpy() hp.setrelheap() hp.setrelheap() # Has to be done twice after the first import (XXX) x=g() print hp.heap() >>> test(g) Partition of a set of 902 objects. Total size = 15364 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 900 100 10800 70 10800 70 int 1 1 0 4128 27 14928 97 list 2 1 0 436 3 15364 100 types.FrameType >>> [ Why it shows an object of type FrameType is something of a mystery, I'm not sure of the best way to get rid of it.] I guess I could add a function somewhat like test(g) to Heapy, depending on what common use cases seem to require. On the other hand, test for memory usage according to alternative (1) would be harder to add, and it is somewhat undefined what it means. And it is perhaps not so useful as the 2nd alternative? Sverker Project home page: http://guppy-pe.sourceforge.net -- http://mail.python.org/mailman/listinfo/python-list