I'm writing a program that processes ~ 25,000 jobs and each iteration
draws a histogram and writes out some of the output. I let it run all
night and when I came back, python was filling up all my memory
(2Gigs) and was thrashing on and off of swap. I narrowed the problem
down to my calling of the hist() function, and was able to reproduce
it in the following code I copied from the mpl website.
Am I not properly closing the figure somehow?
Has this issue already been addressed?
I recently installed version 0.90.1 of matplotlib, although I don't
see any easy way to verify that version number from within python.
-Luke Robison
Code:
-------------------
import os,time,sys
from pylab import *
def report_memory(i):
pid = os.getpid()
a2 = os.popen('ps -p %d -o rss,sz' % pid).readlines()
print i, ' ', a2[1],
return int(a2[1].split()[1])
# take a memory snapshot on indStart and compare it with indEnd
indStart, indEnd = 100, 150
for i in range(indStart,indEnd):
ind = arange(100)
xx = rand(len(ind))
figure(1)
hist(xx)
close(1)
# wait a few cycles for memory usage to stabilize
if i==indStart: start = val
if i>indStart:
end = val
print 'Average memory consumed per loop: %1.4fk bytes' % \
((end-start)/float(indEnd-indStart))
-----------------
Output:
python memtest.py
Average memory consumed per loop: 0.0000k bytes
102 39808 21991
Average memory consumed per loop: 0.0000k bytes
103 39828 21991
Average memory consumed per loop: 0.0000k bytes
104 39852 22024
Average memory consumed per loop: 0.6600k bytes
105 39876 22024
Average memory consumed per loop: 0.6600k bytes
106 39908 22024
Average memory consumed per loop: 0.6600k bytes
107 39932 22024
Average memory consumed per loop: 0.6600k bytes
108 39960 22024
Average memory consumed per loop: 0.6600k bytes
109 39980 22057
Average memory consumed per loop: 1.3200k bytes
110 40008 22057
Average memory consumed per loop: 1.3200k bytes
111 40032 22057
Average memory consumed per loop: 1.3200k bytes
112 40056 22057
Average memory consumed per loop: 1.3200k bytes
113 40084 22057
Average memory consumed per loop: 1.3200k bytes
114 40104 22090
Average memory consumed per loop: 1.9800k bytes
115 40132 22090
Average memory consumed per loop: 1.9800k bytes
116 40156 22090
Average memory consumed per loop: 1.9800k bytes
117 40180 22090
Average memory consumed per loop: 1.9800k bytes
118 40208 22090
Average memory consumed per loop: 1.9800k bytes
119 40232 22123
Average memory consumed per loop: 2.6400k bytes
120 40256 22123
Average memory consumed per loop: 2.6400k bytes
121 40280 22123
Average memory consumed per loop: 2.6400k bytes
122 40304 22123
Average memory consumed per loop: 2.6400k bytes
123 40328 22123
Average memory consumed per loop: 2.6400k bytes
124 40356 22123
Average memory consumed per loop: 2.6400k bytes
125 40380 22156
Average memory consumed per loop: 3.3000k bytes
126 40404 22156
Average memory consumed per loop: 3.3000k bytes
127 40428 22156
Average memory consumed per loop: 3.3000k bytes
128 40452 22156
Average memory consumed per loop: 3.3000k bytes
129 40476 22156
Average memory consumed per loop: 3.3000k bytes
130 40500 22189
Average memory consumed per loop: 3.9600k bytes
131 40528 22189
Average memory consumed per loop: 3.9600k bytes
132 40548 22189
Average memory consumed per loop: 3.9600k bytes
133 40576 22189
Average memory consumed per loop: 3.9600k bytes
134 40596 22189
Average memory consumed per loop: 3.9600k bytes
135 40624 22222
Average memory consumed per loop: 4.6200k bytes
136 40652 22222
Average memory consumed per loop: 4.6200k bytes
137 40676 22222
Average memory consumed per loop: 4.6200k bytes
138 40700 22222
Average memory consumed per loop: 4.6200k bytes
139 40724 22222
Average memory consumed per loop: 4.6200k bytes
140 40744 22222
Average memory consumed per loop: 4.6200k bytes
141 40768 22255
Average memory consumed per loop: 5.2800k bytes
142 40800 22255
Average memory consumed per loop: 5.2800k bytes
143 40824 22255
Average memory consumed per loop: 5.2800k bytes
144 40848 22255
Average memory consumed per loop: 5.2800k bytes
145 40872 22255
Average memory consumed per loop: 5.2800k bytes
146 40896 22288
Average memory consumed per loop: 5.9400k bytes
147 40916 22288
Average memory consumed per loop: 5.9400k bytes
148 40940 22288
Average memory consumed per loop: 5.9400k bytes
149 40972 22288
Average memory consumed per loop: 5.9400k bytes
as you can see, the memory consumption is increasing each loop, and
furthermore, and an increasing rate :-(
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
