Re: [Matplotlib-users] memory usage (leakage?) in ipython interactive mode
Hi Michael, Thanks for your explanation. It turns out that it is a combination of (1) and (3). I hadn't thought about (1) and I hadn't done enough playing to see the python interpreter releasing blocks of memory. As you suggested, the solution is to limit the iPython cache by using the iPython -cs option. thanks for your help, Gary Michael Droettboom wrote: There are at least three possible causes of what you're seeing here: 1) ipython stores references to all results in the console. (ipython maintains a history of results so they can easily be accessed later). I don't recall the details, but it may be possible to turn this feature off or limit the number of objects stored. 2) matplotlib stores references to all figures until they are explicitly closed with pyplot.close(fignum) 3) Python uses pools of memory, and is often imposes a significant delay returning memory to the operating system. It is actually very hard to determine from the outside whether something is leaking or just pooling without compiling a special build of Python with memory pooling turned off. In general, interactive use is somewhat at odds with creating many large plots in a single session, since all of the nice interactive features (history etc.) do not know automagically when the user is done with certain objects. I am not aware of any memory leaks in current versions of matplotlib with *noninteractive* use, other than small leaks caused by bugs in older versions of some of the GUI toolkits (notably gtk+). If you find a script that produces a leak reproducibly, please share so we can track down the cause. Gary Ruben wrote: Doing plot(rand(100)) or matshow(rand(1000,1000)) for example eats a big chunk of memory (tried with TkAgg and WxAgg in Windows (mpl v0.98.5.2) and Linux (mpl v0.98.3)), most of which is not returned when the window is closed. The same goes if you create an array, plot it, and explicitly del it after closing the window. Can you elaborate on these steps? It's possible that the del has little effect, since del only deletes a single reference to the object, not all references which may be keeping it alive (such as the figure, which matplotlib itself keeps a reference to). In general, you need to explicitly call pyplot.close(fignum) to delete a figure. Cheers, Mike -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] memory usage (leakage?) in ipython interactive mode
There are at least three possible causes of what you're seeing here: 1) ipython stores references to all results in the console. (ipython maintains a history of results so they can easily be accessed later). I don't recall the details, but it may be possible to turn this feature off or limit the number of objects stored. 2) matplotlib stores references to all figures until they are explicitly closed with pyplot.close(fignum) 3) Python uses pools of memory, and is often imposes a significant delay returning memory to the operating system. It is actually very hard to determine from the outside whether something is leaking or just pooling without compiling a special build of Python with memory pooling turned off. In general, interactive use is somewhat at odds with creating many large plots in a single session, since all of the nice interactive features (history etc.) do not know automagically when the user is done with certain objects. I am not aware of any memory leaks in current versions of matplotlib with *noninteractive* use, other than small leaks caused by bugs in older versions of some of the GUI toolkits (notably gtk+). If you find a script that produces a leak reproducibly, please share so we can track down the cause. Gary Ruben wrote: Doing plot(rand(100)) or matshow(rand(1000,1000)) for example eats a big chunk of memory (tried with TkAgg and WxAgg in Windows (mpl v0.98.5.2) and Linux (mpl v0.98.3)), most of which is not returned when the window is closed. The same goes if you create an array, plot it, and explicitly del it after closing the window. Can you elaborate on these steps? It's possible that the del has little effect, since del only deletes a single reference to the object, not all references which may be keeping it alive (such as the figure, which matplotlib itself keeps a reference to). In general, you need to explicitly call pyplot.close(fignum) to delete a figure. Cheers, Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] memory usage (leakage?) in ipython interactive mode
Is there a summary somewhere of the current state of knowledge about memory leaks when using the pylab interface interactively? Doing plot(rand(100)) or matshow(rand(1000,1000)) for example eats a big chunk of memory (tried with TkAgg and WxAgg in Windows (mpl v0.98.5.2) and Linux (mpl v0.98.3)), most of which is not returned when the window is closed. The same goes if you create an array, plot it, and explicitly del it after closing the window. I've seen lots of posts over the years about memory leaks, but there's nothing in the FAQ about this. I found old posts about similar things, but nothing that had a clear resolution. thanks, Gary -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
