Hi,
so here is some quick but working example. I added there are 2-3 functions
(unused)
as a bonus, you can easily call them from the main function using same API
(except the piechart). I hope this shows what I lack in matplotlib - a general
API
so that I could easily switch form scatter plot to piechart or barchart without
altering
much the function arguments. Messing with return objects line2D,
PathCollection, Rectangle
is awkward and I would like to stay away from matplotlib's internals. ;) Some
can be sliced,
so not, you will see in the code.
This eatmem.py will take easily all your memory. Drawing 300000 dots is not
feasible
with 16GB of RAM. While the example is for sure inefficient in many places
generating the data
in python does not eat RAM. That happens afterwards.
I would really like to hear whether matplotlib could be adjusted instead. ;) I
already mentioned
in this thread that it is awkward to pre-create colors before passing all data
to a drawing
function. I think we could all save a lot if matplotlib could dynamically fetch
colors
on the fly from user-created generator, same for legends descriptions. I think
my example
code shows the inefficient approach here. Would I have more time I would
randomize a bit
more the sublist of each series so that the numbers in legends would be more
variable
but that is a cosmetic issue.
Probably due to my ignorance you will see that figures with legends have
different font
sizes, axes are rescaled and the figure. Of course I wanted to have the drawing
same via both
approaches but failed badly. The files/figures with legends should be just
accompanied by the
legend "table" underneath but the drawing itself should be same. Maybe an issue
with DPI settings
but not only.
I placed some comments in the code, please don't take them in person. ;) Of
course
I am glad for the existing work and am happy to contribute my crap. I am fine
if you rewamp
this ugly code into matplotlib testsuite, provide similar function (the API
mentioned above)
so that I could use your code directly. That would be great. I just tried to
show multiple
issues at once, notably that is why I included those unused functions. You will
for sure find
a way to use them.
Regarding the "unnecessary" del() calls etc., I think I have to use keep some,
Ben, because
the function is not always left soon enough. I could drop some, you are right,
but for some
I don't think so. Matplotlib cannot recycle the memory until me (upstream)
deletes the reference
so ... go and test this lousy code. Now you have a testcase. ;) Same with the
gc.collect() calls.
Actually, the main loop with 10 iteration is there just to show why I always
want to clear
a figure when entering a function and while leaving it as well. It happened too
many times that
I drawed over an old figure, and this was posted also few times on this list by
others. That is
a weird behavior in my opinion. We, users, are just forced to use too low-level
functions.
So, have fun eating your memory! :))
Martin
#! /usr/bin/env python
import sys
import gc
from textwrap import wrap
from itertools import izip, imap, ifilter, chain
import numpy as np
from math import ceil
import colorsys
import matplotlib
matplotlib.use('Agg')
# Force matplotlib not to use any X-windows backend.
import pylab
matplotlib.use('Agg')
from random import uniform, randint, randrange
from optparse import OptionParser
myversion = 'xxx'
myparser = OptionParser(version="%s version %s" % ('%prog', myversion))
myparser.add_option("--series-num", action="store", type="int", dest="series_num", default=200,
help="Set number of series in the charts. Each series has its own color and legend text.")
myparser.add_option("--max-datapoints-per-series", action="store", type="int", dest="max_datapoints_per_series", default=2000,
help="Set number of data points to be generated at random. The actual counts will appear in the legend.")
(myoptions, myargs) = myparser.parse_args()
# convert the view of numpy array to tuple
# http://matplotlib.1069221.n5.nabble.com/RendererAgg-int-width-int-height-dpi-debug-False-ValueError-width-and-height-must-each-be-below-32768-td27756.html
def generate_color_tuples(cnt, start, stop):
_HSV_tuples1 = []
for _n in xrange(1,cnt + 1):
_h1 = sorted([uniform(start, stop) for x in xrange(_n)])
_HSV_tuples1 = [(_h1[x], 1.0, 1.0) for x in xrange(_n)]
return [colorsys.hsv_to_rgb(*x) for x in _HSV_tuples1]
def generate_color_tuples_wrapper(_wanted_length):
"""Generating lots of colors is useless. Try to make a color list using batches of
colors. Make 100 different colors and then re-use them to get the final number.
"""
if _wanted_length > 300:
_manageable_length = ( _wanted_length / 100 ) + 1 # round up
_short_colors = generate_color_tuples(100, 0.01, 0.95) # 0.01, 0.95)
_colors = []
# this way we rotate the color batches several times
#for _i in xrange(1, _manageable_length+1):
# print "_i = %s, _manageable_length=%s" % (_i, _manageable_length)
# _colors += _short_colors
# this way we rather keep one color for 100 items, then move to next color
# in the whole series will use all 100 colors and then use the very first color
# if few entries need it
for _c in _short_colors:
_colors += _manageable_length * [_c]
_missing_cnt = _wanted_length - len(_colors)
if _missing_cnt > 0:
_colors += _short_colors[:_missing_cnt]
else:
_colors = generate_color_tuples(_wanted_length, 0.01, 0.95) # 0.01, 0.95)
return _colors
def get_ncol(legend, fontsize=8, chars_per_line=68, rows_per_subfigure_8ptsize=47, rows_per_subfigure_10ptsize=35):
"""Determine lengths of each legend text. Two-columns fit onto a page if every legend text is shorter than about 68 chars.
47 rows fit into the subplot 212 when fontsize is 8.
TODO: improve the code to decrease fontsize or enlarge figure size if more than 47 rows are to be necessary.
"""
if not legend:
raise ValueError, "get_ncol(): legend list is empty, no way to calculate how many columns and subfigures and axes are necessary"
# get lengths of individual items
_lengths = map(lambda x: len(x), legend)
# get max number of columns
if filter(lambda x: x > chars_per_line, _lengths):
_max_columns = 1
elif filter(lambda x: x > (chars_per_line / 2), _lengths):
_max_columns = 2
elif filter(lambda x: x > (chars_per_line / 3), _lengths):
_max_columns = 3
else:
_max_columns = 4
if fontsize == 8:
_need_columns_per_subfigure = ceil ( float(len(legend)) / rows_per_subfigure_8ptsize ) # round up
elif fontsize == 10:
_need_columns_per_subfigure = ceil ( float(len(legend)) / rows_per_subfigure_10ptsize )
else:
_need_columns_per_subfigure = ceil ( float(len(legend)) / rows_per_subfigure_10ptsize )
_need_subfigures = ceil ( float(_need_columns_per_subfigure) / float(_max_columns) ) + 1
if not ceil(_need_subfigures):
_need_subfigures = 1
if _need_subfigures < 3:
_ax1 = 211
_ax2 = 212
elif _need_subfigures < 4:
_ax1 = 311
_ax2 = 312
elif _need_subfigures < 5:
_ax1 = 411
_ax2 = 412
elif _need_subfigures < 6:
_ax1 = 511
_ax2 = 512
elif _need_subfigures < 6:
_ax1 = 511
_ax2 = 512
elif _need_subfigures < 7:
_ax1 = 611
_ax2 = 612
elif _need_subfigures < 8:
_ax1 = 711
_ax2 = 712
elif _need_subfigures < 9:
_ax1 = 811
_ax2 = 812
else:
_ax1 = 911
_ax2 = 912
# https://github.com/matplotlib/matplotlib/issues/2098
# For higher number of subfigures use plt.subplot(13, 1, 1) instead. See The documentation for subplot (http://matplotlib.org/api/pyplot_api.html?highlight=subplot#matplotlib.pyplot.subplot) does state that the maximum number of rows/columns/plot_number is 9. The problem is that anything other than three digits is ill-defined. The solution is to use the better approach of plt.subplot(13, 1, 1).
return ceil(_need_subfigures), _ax1, _ax2, _max_columns # round up to ensure there is better some white then a truncated legend
def set_my_pylab_defaults():
params = {
'axes.labelsize': 10,
'text.fontsize': 10,
'legend.fontsize': 8,
'xtick.labelsize': 10,
'ytick.labelsize': 10,
'font.serif': 'Bitstream Vera Sans'
}
pylab.rcParams.update(params)
def set_limits(ax, xmin=None, xmax=None, ymin=None, ymax=None):
if xmin is not None:
ax.set_xlim(xmin=xmin)
if xmax is not None:
ax.set_xlim(xmax=xmax)
if ymin is not None:
ax.set_ylim(ymin=ymin)
if ymax is not None:
ax.set_ylim(ymax=ymax)
def draw_scatter_plot(filename, mydata_x, mydata_y, colors, title_data, xlabel_data, ylabel_data, legends, legend_loc='upper right', legend_bbox_to_anchor=(1.0, 1.0), legend_ncol=None, xmin=None, xmax=None, ymin=None, ymax=None, x_logscale=False, y_logscale=False, fontsize=10, legend_fontsize=8, dpi=100, tight_layout=False, legend_inside=False, objsize=0.2):
"""A wrapper function to a draw a scatter plot and allowing to set multiple columns
in the text legend, to draw legend inside the figure or below the plot as another
subplot. The height of the figure is dynamically calculated when the legend columns
do not fit into the image. However, it seems maximum height of the PNG image is about
200 inches, definitely around 5xx matplotlib is crashing.
The title line is dynamically wrapped using '\n' as necessary while using most of the
white space on each line. Therefore, the least number of rows is used for a title.
"""
if len(mydata_x) != len(mydata_y):
raise ValueError, "%s: len(mydata_x) != len(mydata_y): %s != %s" % (filename, len(mydata_x), len(mydata_y))
if colors and len(mydata_x) != len(colors):
sys.stderr.write("Warning: draw_scatter_plot(): %s: len(mydata_x) != len(colors): %s != %s.\n" % (filename, len(mydata_x), len(colors)))
if colors and legends and len(colors) != len(legends):
sys.stderr.write("Warning: draw_scatter_plot(): %s: len(colors) != len(legends): %s != %s.\n" % (filename, len(colors), len(legends)))
sys.stderr.write("Debug: draw_scatter_plot(): filename='%s', len(mydata_x)=%s, len(mydata_y)=%s, len(colors)=%s, title_data='%s', xlabel_data='%s', ylabel_data='%s', len(legends)=%s\n" % (filename, len(mydata_x), len(mydata_y), len(colors), title_data.rstrip('\n'), str(xlabel_data), str(ylabel_data), len(legends)))
if mydata_x and mydata_y and filename:
if legends:
if not legend_ncol:
_subfigs, _ax1_num, _ax2_num, _legend_ncol = get_ncol(legends, fontsize=legend_fontsize)
else:
_subfigs, _ax1_num, _ax2_num, _legend_ncol = 2, 211, 212, legend_ncol
else:
_subfigs, _ax1_num, _legend_ncol = 1, 111, 0
set_my_pylab_defaults()
pylab.clf()
_figure = pylab.figure()
_figure.clear()
_figure.set_tight_layout(True)
if legends:
if 8.4 * _subfigs < 200:
_figure.set_size_inches(11.2, 8.4 * _subfigs)
else:
# _figure.set_size_inches() silently accepts a large value but later on _figure.savefig() crashes with:
# ValueError: width and height must each be below 32768
_figure.set_size_inches(11.2, 200)
sys.stderr.write("Warning: draw_scatter_plot(): Wanted to set %s figure height to %s but is too high, forcing %s instead.\n" % (filename, 8.4 * _subfigs, 200))
print "Debug: draw_scatter_plot(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
_ax1 = _figure.add_subplot(_ax1_num)
_ax2 = _figure.add_subplot(_ax2_num)
else:
_figure.set_size_inches(11.2, 8.4)
_ax1 = _figure.gca()
print "Debug: draw_scatter_plot(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
if x_logscale:
_ax1.set_yscale('log')
if y_logscale:
_ax1.set_xscale('log')
if legends:
_series = [_ax1.scatter(_x, _y, color=_c, s=objsize, label=_l) for _x, _y, _c, _l in izip(mydata_x, mydata_y, colors, legends)] # returns PathCollection object
_ax2.legend(_series, legends, loc='upper left', bbox_to_anchor=(0,0,1,1), borderaxespad=0., ncol=_legend_ncol, mode='expand', fontsize=legend_fontsize)
del(_series)
_ax2.set_frame_on(False)
_ax2.tick_params(bottom='off', left='off', right='off', top='off')
pylab.setp(_ax2.get_yticklabels(), visible=False)
pylab.setp(_ax2.get_xticklabels(), visible=False)
else:
# collect the scatter objects so that they do not live (until this function exists) in the wild and we can delete them immediately
_series = [_ax1.scatter(_x, _y, color=_c, s=objsize) for _x, _y, _c in izip(mydata_x, mydata_y, colors)] # returns PathCollection object
del(_series)
_ax1.set_xlabel(xlabel_data, fontsize=fontsize)
_ax1.set_ylabel(ylabel_data, fontsize=fontsize)
set_limits(_ax1, xmin, xmax, ymin, ymax)
if fontsize == 10:
_ax1.set_title('\n'.join(wrap(title_data, 100)), fontsize=fontsize+2)
elif fontsize == 12:
_ax1.set_title('\n'.join(wrap(title_data, 90)), fontsize=fontsize+2)
else:
_ax1.set_title('\n'.join(wrap(title_data, 100)), fontsize=fontsize+2)
if legends:
_figure.savefig(filename, dpi=100) #, bbox_inches='tight')
del(_ax2)
else:
_figure.savefig(filename, dpi=100)
del(_ax1)
_figure.clear()
del(_figure)
pylab.clf()
pylab.close()
# pylab.rcdefaults()
gc.collect()
def draw_hist2d_plot(filename, mydata_x, mydata_y, colors, title_data, xlabel_data, ylabel_data, legends, legend_loc='upper right', legend_bbox_to_anchor=(1.0, 1.0), legend_ncol=None, xmin=None, xmax=None, ymin=None, ymax=None, fontsize=10, legend_fontsize=8, dpi=100, tight_layout=False, legend_inside=False, objsize=0.1):
"""The name of this function is silly, is just a bit different scatter plot.
"""
# hist2d(x, y, bins = None, range=None, weights=None, cmin=None, cmax=None **kwargs)
# matplotlib in some cases dies if there is e.g. one color more than data and legend items.
# That is unhelpful if one generates colors automagically, sometimes it is just easier to have a
# bit more and let matplotlib use as much as it needs to. Here some debug messages for us
# before matplolib dies with an unhelpful message. ;)
if len(mydata_x) != len(mydata_y):
raise ValueError, "%s: len(mydata_x) != len(mydata_y): %s != %s" % (filename, len(mydata_x), len(mydata_y))
if colors and len(mydata_x) != len(colors):
sys.stderr.write("Warning: draw_hist2d_plot(): %s: len(mydata_x) != len(colors): %s != %s.\n" % (filename, len(mydata_x), len(colors)))
if colors and legends and len(colors) != len(legends):
sys.stderr.write("Warning: draw_hist2d_plot(): %s, len(colors) != len(legends): %s != %s.\n" % (filename, len(colors), len(legends)))
sys.stderr.write("Debug: draw_hist2d_plot(): filename='%s', len(mydata_x)=%s, len(mydata_y)=%s, len(colors)=%s, title_data='%s', xlabel_data='%s', ylabel_data='%s', len(legends)=%s\n" % (filename, len(mydata_x), len(mydata_y), len(colors), title_data.rstrip('\n'), str(xlabel_data), str(ylabel_data), len(legends)))
if mydata_x and mydata_y and filename:
if legends:
if not legend_ncol:
_subfigs, _ax1_num, _ax2_num, _legend_ncol = get_ncol(legends, fontsize=legend_fontsize)
else:
_subfigs, _ax1_num, _ax2_num, _legend_ncol = 3, 213, 313, legend_ncol
else:
_subfigs, _ax1_num, _legend_ncol = 3, 313, 0
set_my_pylab_defaults()
pylab.clf()
_figure = pylab.figure()
_figure.clear()
_figure.set_tight_layout(True)
gc.collect()
if legends:
if 8.4 * _subfigs < 200:
_figure.set_size_inches(11.2, 8.4 * (_subfigs + 1))
else:
# _figure.set_size_inches() silently accepts a large value but later on _figure.savefig() crashes with:
# ValueError: width and height must each be below 32768
_figure.set_size_inches(11.2, 200)
sys.stderr.write("Warning: draw_hist2d_plot(): Wanted to set %s figure height to %s but is too high, forcing %s instead.\n" % (filename, 8.4 * _subfigs, 200))
print "Debug: draw_hist2d_plot(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
_ax1 = _figure.add_subplot(_ax1_num)
_ax2 = _figure.add_subplot(_ax2_num)
else:
_figure.set_size_inches(22.4, 8.4 * 4) # make it more high-resolution, the DPI way does not help much
_ax1 = _figure.gca()
print "Debug: draw_hist2d_plot(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
if legends:
_series = [_ax1.scatter(_x, _y, color=_c, s=objsize, label=_l, hatch='.') for _x, _y, _c, _l in izip(mydata_x, mydata_y, colors, legends)] # returns PathCollection object
# TODO: improve matplotlib documentation
# want to force a rectangle filled with the appropriate color in the legend instead of the tiny 3 dots (hatch), can hardly see them
_ax2.legend(_series, legends, loc='upper left', bbox_to_anchor=(0,0,1,1), borderaxespad=0., ncol=_legend_ncol, mode='expand', fontsize=legend_fontsize)
_ax2.set_frame_on(False)
_ax2.tick_params(bottom='off', left='off', right='off', top='off')
pylab.setp(_ax2.get_yticklabels(), visible=False)
pylab.setp(_ax2.get_xticklabels(), visible=False)
else:
_series = [_ax1.scatter(_x, _y, color=_c, s=objsize) for _x, _y, _c in izip(mydata_x, mydata_y, colors)] # returns PathCollection object
del(_series)
gc.collect()
_ax1.set_xlabel(xlabel_data, fontsize=fontsize)
_ax1.set_ylabel(ylabel_data, fontsize=fontsize)
_ax1.set_xmargin(0.05)
_ax1.set_ymargin(0.05)
set_limits(_ax1, xmin, xmax, ymin, ymax)
if fontsize == 10:
_ax1.set_title('\n'.join(wrap(title_data, 100)), fontsize=fontsize+2)
elif fontsize == 12:
_ax1.set_title('\n'.join(wrap(title_data, 90)), fontsize=fontsize+2)
else:
_ax1.set_title('\n'.join(wrap(title_data, 100)), fontsize=fontsize+2)
gc.collect()
if legends:
_figure.savefig(filename, dpi=150) #, bbox_inches='tight')
del(_ax2)
else:
_figure.savefig(filename, dpi=150)
del(_ax1)
_figure.clear()
del(_figure)
pylab.clf()
pylab.close()
# pylab.rcdefaults()
gc.collect()
def add_counts_to_legend_items(legends, indexes):
"""Modify existing legend list and add a number of occurrences to each legend item by
counting len(indexes[someitem]).
"""
if legends:
_legends = []
for _cnt, _legend in izip(imap(lambda someitem: len(someitem), indexes), legends):
_legends.append('%dx %s' % (_cnt, _legend))
else:
_legends = legends
return _legends
def draw_hist_plot(filename, mydata, colors, title_data, xlabel_data, ylabel_data, legends, legend_loc='upper right', legend_bbox_to_anchor=(1.0, 1.0), legend_ncol=None, bin_count=50, align_pos='mid', log_type=False, xmin=None, xmax=None, ymin=None, ymax=None, fontsize=10, legend_fontsize=8, dpi=100, tight_layout=False, legend_inside=False):
"""http://matplotlib.org/users/tight_layout_guide.html
For some reason, when addition of the legend makes the image taller but also, increases Y-axis length
and make axis fonts taller and also smaller. The code below probably adjusts the figure size incorrectly
but at least, it can dynamically resize the figure so that the legend fits into the figure.
"""
sys.stderr.write("Debug: draw_hist_plot(): filename='%s', len(mydata)=%s, len(colors)=%s, title_data='%s', xlabel_data='%s', ylabel_data='%s', len(legends)=%s\n" % (filename, len(mydata), len(colors), str(title_data[:-1]), str(xlabel_data), str(ylabel_data), len(legends)))
if colors and len(mydata) != len(colors) and len(colors) != 1:
sys.stderr.write("Warning: draw_hist_plot(): %s: len(mydata) != len(colors): %s != %s.\n" % (filename, len(mydata), len(colors)))
if colors and legends and len(colors) != len(legends) and len(colors) != 1:
sys.stderr.write("Warning: draw_hist_plot(): %s: len(colors) != len(legends): %s != %s.\n" % (filename, len(colors), len(legends)))
if mydata and filename:
if legends:
if not legend_ncol:
_subfigs, _ax1_num, _ax2_num, _legend_ncol = get_ncol(legends, fontsize=legend_fontsize)
else:
_subfigs, _ax1_num, _ax2_num, _legend_ncol = 2, 211, 212, legend_ncol
else:
_subfigs, _ax1_num, _ax2_num = 1, 111, 111
if not legend_ncol and not legend_inside:
_legend_ncol = 0
else:
_legend_ncol = legend_ncol
set_my_pylab_defaults()
pylab.clf()
_figure = pylab.figure()
_figure.clear()
_figure.set_tight_layout(True)
if legends and not legend_inside:
if 8.4 * _subfigs < 200:
_figure.set_size_inches(11.2, 8.4 * _subfigs)
else:
# _figure.set_size_inches() silently accepts a large value but later on _figure.savefig() crashes with:
# ValueError: width and height must each be below 32768
# once needed even 1394.4!
_figure.set_size_inches(11.2, 200)
sys.stderr.write("Warning: draw_hist_plot(): Wanted to set %s figure height to %s but is too high, forcing %s instead.\n" % (filename, 8.4 * _subfigs, 200))
print "Debug: draw_hist_plot(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
_ax1 = _figure.add_subplot(_ax1_num)
_ax2 = _figure.add_subplot(_ax2_num)
#sys.stderr.write("mydata=%s\n" % str(mydata))
ret = _ax1.hist(mydata, histtype='barstacked', bins=bin_count, align='mid', color=colors, log=log_type, label=legends)
# sys.stderr.write("ret=%s\n" % str(ret))
if len(ret):
try:
reclist = [patchlist[0] for patchlist in ret[2]]
except TypeError: # TypeError: 'Rectangle' object does not support indexing
reclist = ret[2]
else:
sys.stderr.write("Error: %s: no legend data to be moved from _ax1 under _ax2. mydata=%s\n" % (filename, str(mydata)))
#sys.stderr.write("reclist=%s\n" % str(reclist))
_ax2.legend(reclist, legends, loc='upper left', bbox_to_anchor=(0,0,1,1), borderaxespad=0., ncol=_legend_ncol, mode='expand', fontsize=legend_fontsize)
_ax2.set_frame_on(False)
_ax2.tick_params(bottom='off', left='off', right='off', top='off')
pylab.setp(_ax2.get_yticklabels(), visible=False)
pylab.setp(_ax2.get_xticklabels(), visible=False)
else:
_figure.set_size_inches(11.2, 8.4)
_ax1 = _figure.gca()
print "Debug: draw_hist_plot(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
ret = _ax1.hist(mydata, histtype='barstacked', bins=bin_count, align='mid', color=colors, label=legends, log=log_type)
if legend_inside:
try:
_median = np.median(mydata)
except:
_median = np.median(map(lambda x: np.median(x), mydata))
try:
reclist = [patchlist[0] for patchlist in ret[2]]
except TypeError: # TypeError: 'Rectangle' object does not support indexing
reclist = ret[2]
if _median < 600:
_ax1.legend(reclist, legends, loc='upper right', bbox_to_anchor = (1.0, 1.0), ncol=1)
else:
_ax1.legend(reclist, legends, loc='upper left', bbox_to_anchor = (0, 1.0), ncol=1)
_ax1.set_xlabel(xlabel_data, fontsize=fontsize)
_ax1.set_ylabel(ylabel_data, fontsize=fontsize)
set_limits(_ax1, xmin, xmax, ymin, ymax)
if fontsize == 10:
_ax1.set_title('\n'.join(wrap(title_data, 100)), fontsize=fontsize+2)
elif fontsize == 12:
_ax1.set_title('\n'.join(wrap(title_data, 90)), fontsize=fontsize+2)
else:
_ax1.set_title('\n'.join(wrap(title_data, 100)), fontsize=fontsize+2)
if legends and not legend_inside:
_figure.savefig(filename, dpi=100) #, bbox_inches='tight')
del(_ax2)
del(ret)
del(reclist)
else:
if tight_layout:
_figure.savefig(filename, dpi=100, bbox_inches='tight')
else:
_figure.savefig(filename, dpi=100)
del(_ax1)
_figure.clear()
del(_figure)
pylab.clf()
pylab.close()
# pylab.rcdefaults()
gc.collect()
def draw_piechart(filename, title, data, colors, labels, labeldistance=1.5, autopct='%1.1f%%', fontsize=12):
"""By default matplotlib enforces uneven X and Y lengths and therefore a circle appears
as an ellipse instead. Therefore fix it here by forcing equal X- and Y-axis lengths to obtain the circle.
Descriptions/legends of pie slices are horizontally aligned. If there are many pie slices
with descriptions their texts overlap. That would be less a problem is the legend text was rotated in the
same angle as axis of it's respective slice: https://github.com/matplotlib/matplotlib/issues/2304
"""
sys.stderr.write("Debug: draw_piechart(): filename='%s', len(data)=%s, len(colors)=%s, title='%s', labels='%s'\n" % (filename, len(data), len(colors), title.rstrip('\n'), str(labels)))
if data and pylab and filename:
pylab.clf()
_figure = pylab.figure()
_figure.set_tight_layout(False)
if filter(lambda x: len(x) > 45, labels):
_figure.set_size_inches(30, 30)
else:
_figure.set_size_inches(25, 25)
print "Debug: draw_piechart(): Changed %s figure size to: %s" % (filename, str(_figure.get_size_inches()))
_ax_pie = pylab.axes([0.30, 0.30, 0.4, 0.4])
_ax_pie.pie(data, colors=colors, labels=labels, pctdistance=0.75, labeldistance=labeldistance, autopct=autopct)
pylab.axis('equal')
pylab.title('\n'.join(wrap(title, 100)), y=-0.5)
_figure.savefig(filename, dpi=100)
del(_ax_pie)
_figure.clear()
pylab.clf()
# release the memory, thanks to
# /usr/lib64/python2.7/site-packages/matplotlib/pyplot.py:412: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_num_figures`).
pylab.close()
# pylab.rcdefaults()
gc.collect()
def main():
"""The following takes at least 15.3GB of RAM: python ./eatmem.py --series-num=300000 --max-datapoints-per-series=3
"""
_data_x = [ [randrange(2550,7500) for y in xrange(myoptions.max_datapoints_per_series) ] for x in xrange(0,myoptions.series_num)]
_data_y = [ [randrange(720,1500) for y in xrange(myoptions.max_datapoints_per_series) ] for x in xrange(0,myoptions.series_num)]
# cannot use a colorbar for the dataset as the colors are not contiguous, however
# _colors = [(0.48, 0.2, 0.7) for x in xrange(0,myoptions.series_num)]
_colors = generate_color_tuples_wrapper(myoptions.series_num)
_legends = ['mytext ds s s' for x in xrange(0,myoptions.series_num)]
# make some items more distinct from the crowd
_extra_colors = generate_color_tuples(10, 0.1, 0.8)
for _a, _i in enumerate([1, 3, 7, 30, 40, 99, 105, 999, 1000, 5000]):
if _i < myoptions.series_num:
_colors[_i] = _extra_colors[_a]
_legends[_i] = 'a special item is here'
_data_x[_i] = _data_x[_i][:10] # make them having less values
_data_y[_i] = _data_y[_i][:10] # make them having less values
_data_x[_i-5] = _data_x[_i-5][:2] # and randomize a bit some other as well
_data_y[_i-5] = _data_y[_i-5][:2] # and randomize a bit some other as well
_legends = add_counts_to_legend_items(_legends, _data_x)
for _iteration in xrange(1,30):
print "Debug: main(): Going to render figure %s" % _iteration
_title = "This is really long title which needs to be wrapped manually before feeding into matplotlib drawing function. Sometimes the figure overlaps the title so make sure there are some newlines forced right after the title text to prevent the figure overwriting it. The figure contains %d series with maximum %d items per each.\n\n\n\n" % (myoptions.series_num, myoptions.max_datapoints_per_series)
_xlabel = "X coordinate position"
_ylabel = "Y coordinate position"
_legend_loc = 'upper left'
_legend_bbox_to_anchor = (-0.1, -0.1)
draw_hist2d_plot('eatmem_with_legend_' + str(_iteration) + '.png', _data_x, _data_y, _colors, _title, _xlabel, _ylabel, _legends, xmin=None, xmax=None, ymin=0, ymax=None, fontsize=10, dpi=300, objsize=0.5)
draw_hist2d_plot('eatmem_without_legend_' + str(_iteration) + '.png', _data_x, _data_y, _colors, _title, _xlabel, _ylabel, [], xmin=None, xmax=None, ymin=None, ymax=None, fontsize=10, dpi=300, objsize=0.5)
if __name__ == "__main__":
main()
# vim:ts=4:sw=4:expandtab:smartindent
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
