Benjamin Root wrote: > > > > On Thu, Oct 10, 2013 at 9:05 AM, Martin MOKREJŠ <mmokr...@gmail.com > <mailto:mmokr...@gmail.com>> wrote: > > Hi, > rendering some of my charts takes almost 50GB of RAM. I believe below > is a stracktrace > of one such situation when it already took 15GB. Would somebody comments > on what is > matplotlib doing at the very moment? Why the recursion? > > The charts had to have 262422 data points in a 2D scatter plot, each > point has assigned > its own color. They are in batches so that there are 153 distinct colors > but nevertheless, > I assigned to each data point a color value. There are 153 legend items > also (one color > won't be used). > > ^CTraceback (most recent call last): > ... > _figure.savefig(filename, dpi=100) > File "/usr/lib64/python2.7/site-packages/matplotlib/figure.py", line > 1421, in savefig > self.canvas.print_figure(*args, **kwargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/backend_bases.py", > line 2220, in print_figure > **kwargs) > File > "/usr/lib64/python2.7/site-packages/matplotlib/backends/backend_agg.py", line > 505, in print_png > FigureCanvasAgg.draw(self) > File > "/usr/lib64/python2.7/site-packages/matplotlib/backends/backend_agg.py", line > 451, in draw > self.figure.draw(self.renderer) > File "/usr/lib64/python2.7/site-packages/matplotlib/artist.py", line > 54, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/figure.py", line > 1034, in draw > func(*args) > File "/usr/lib64/python2.7/site-packages/matplotlib/artist.py", line > 54, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/axes.py", line > 2086, in draw > a.draw(renderer) > File "/usr/lib64/python2.7/site-packages/matplotlib/artist.py", line > 54, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/collections.py", > line 718, in draw > return Collection.draw(self, renderer) > File "/usr/lib64/python2.7/site-packages/matplotlib/artist.py", line > 54, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/collections.py", > line 276, in draw > offsets, transOffset, self.get_facecolor(), self.get_edgecolor(), > File "/usr/lib64/python2.7/site-packages/matplotlib/collections.py", > line 551, in get_edgecolor > return self._edgecolors > KeyboardInterrupt > ^CError in atexit._run_exitfuncs: > Traceback (most recent call last): > File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/_pylab_helpers.py", > line 90, in destroy_all > gc.collect() > KeyboardInterrupt > Error in sys.exitfunc: > Traceback (most recent call last): > File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/usr/lib64/python2.7/site-packages/matplotlib/_pylab_helpers.py", > line 90, in destroy_all > gc.collect() > KeyboardInterrupt > > ^C > > > Clues what is the code doing? I use mpl-1.3.0. > Thank you, > Martin > > > Unfortunately, that stacktrace isn't very useful. There is no recursion > there, but rather the perfectly normal drawing of the figure object that has > a child axes, which has child collections which have child artist objects. > > Without the accompanying code, it would be difficult to determine where the > memory hog is.
Could there be places where gc.collect() could be introduced? Are there places where matplotlib could del() unnecessary objects right away? I think the problem is with huge lists or pythonic dicts. I could save 10GB of RAM when I converted one python dict to a bsddb3 file having just 10MB on disk. I speculate matplotlib in that code keeps the data in some huge list or more likely a dict and that is the same issue. Are you sure you cannot see where a problem is? It happens (is visible) only with huge number of dots, of course. Thanks, Martin ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk _______________________________________________ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
