Re: [matplotlib-devel] performance (speed) of logarithmic plots
On Thu, Mar 18, 2010 at 6:21 PM, Andrew Hawryluk wrote:
> I've observed a significant difference in the time required by different
> plotting functions. With a plot of 5000 random data points (all
> positive, non-zero), plt.semilogx takes 3.5 times as long as plt.plot.
> (Data for the case of saving to PDF, ratio changes to about 3.1 for PNG
> on my machine.)
>
> I used cProfile (script attached) and found several significant
> differences between the profiles of each plotting command. On my first
> analysis, it appears that most of the difference is due to increased use
> of mathtext in semilogx:
>
>==
>Plotting command
> ==
> cumtime (s) plotsemilogx semilogy loglog
> ==
> total running time 0.618 2.192 0.953 1.362
> axis.py:181(draw) 0.118 1.500 0.412 0.569
> text.py:504(draw) 0.056 1.353 0.290 0.287
> mathtext.py:2765(__init__) 0.000 1.018 0.104 0.103
> mathtext.py:2772(parse) --- 1.294 0.143 0.254
> pyparsing.py:1018(parseString) --- 0.215 0.216 0.221
> pyparsing.py:3129(oneOf)--- 0.991 --- ---
> pyparsing.py:3147() --- 0.358 --- ---
> lines.py:918(_draw_solid) 0.243 0.358 0.234 0.352
> =
>
> It seems that semilogx could be made as fast as semilogy since they have
> to do the same amount of work, but I'm not sure where the differences
> lie. Can anyone suggest where I should look first?
>
> Much thanks,
>
> Andrew Hawryluk
>
> matplotlib.__version__ = '0.99.1'
> Windows XP Professional
> Version 2002, Service Pack 3
> Intel Pentium 4 CPU 3.00 GHz, 2.99 GHz, 0.99 GB of RAM
>
>
>
Hello,
How did you get the cumtime listing? The output of the run doesn't produce a
cumulative sum table as you showed here.
Platform :
Linux-2.6.31.9-174.fc12.i686.PAE-i686-with-fedora-12-Constantine
Python : ('CPython', 'tags/r262', '71600')
NumPy: 1.5.0.dev8038
Matplotlib : 1.0.svn
--
Gökhan
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] performance (speed) of logarithmic plots
> Hello, > How did you get the cumtime listing? The output of the run doesn't produce a > cumulative sum table as you showed here. > Gökhan No, it doesn't. The output of the run is four huge cProfile listings, one for each plotting command tested. I manually searched the data for long cumtime's that differed between the plots and typed the table myself. I have also confirmed the speed differences on matplotlib 0.99.0 under Ubuntu 9.10: plot 0.629 CPU seconds semilogx 3.430 CPU seconds semilogy 1.044 CPU seconds loglog1.479 CPU seconds I'll try to figure out why semilogx uses so much more mathtext than semilogy, but if anyone familiar with the code is curious enough to look into it they will probably beat me to the answer. Andrew -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] performance (speed) of logarithmic plots
This is indeed a very interesting result and I am able to reproduce similar ratios for total running time. However, I think the semilogx result is somewhat of a red herring. If you change the order of the tests in your script, you'll notice that the first "*log*" plot always takes the longest run time. If you run each test in a separate process, all of the "*log*" run times are approximately equal (with loglog being slightly slower). The reason for this is the caching of mathtext expressions. I agree that mathtext is the bottleneck -- but mathtext expressions are only parsed and rendered the first time they are encountered, and simply pulled from a cache after that. It's sort of a "known issue" that mathtext is slow-ish. It's a very function-call heavy and object-oriented bit of code and most attempts at optimization seem to lead to too much uglification. The algorithms themselves are from TeX, so I don't know if there's much room for improvement, but there is something about the translation from Pascal/C to Python that creates a very different performance profile. An interesting result may be to disable the mathtext rendering for log plots (by setting the axis formatters to something static) and comparing those numbers. That would give a better sense of the overhead of merely log-transforming the points and the transformation system itself. I don't think a factor of 2 is too problematic, given all of the extra work that has to be done to maintain two copies of the data, extra care to calculate xlim and ylim etc. Mike Andrew Hawryluk wrote: > I've observed a significant difference in the time required by different > plotting functions. With a plot of 5000 random data points (all > positive, non-zero), plt.semilogx takes 3.5 times as long as plt.plot. > (Data for the case of saving to PDF, ratio changes to about 3.1 for PNG > on my machine.) > > I used cProfile (script attached) and found several significant > differences between the profiles of each plotting command. On my first > analysis, it appears that most of the difference is due to increased use > of mathtext in semilogx: > > == > Plotting command > == > cumtime (s) plotsemilogx semilogy loglog > == > total running time 0.618 2.192 0.953 1.362 > axis.py:181(draw) 0.118 1.500 0.412 0.569 > text.py:504(draw) 0.056 1.353 0.290 0.287 > mathtext.py:2765(__init__) 0.000 1.018 0.104 0.103 > mathtext.py:2772(parse) --- 1.294 0.143 0.254 > pyparsing.py:1018(parseString) --- 0.215 0.216 0.221 > pyparsing.py:3129(oneOf)--- 0.991 --- --- > pyparsing.py:3147() --- 0.358 --- --- > lines.py:918(_draw_solid) 0.243 0.358 0.234 0.352 > = > > It seems that semilogx could be made as fast as semilogy since they have > to do the same amount of work, but I'm not sure where the differences > lie. Can anyone suggest where I should look first? > > Much thanks, > > Andrew Hawryluk > > matplotlib.__version__ = '0.99.1' > Windows XP Professional > Version 2002, Service Pack 3 > Intel Pentium 4 CPU 3.00 GHz, 2.99 GHz, 0.99 GB of RAM > > > > -- > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > > > ___ > Matplotlib-devel mailing list > Matplotlib-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
[matplotlib-devel] Elusive bug plotting masked arrays
I am trying to plot two 1-D masked arrays against each other in a line plot and an extraneous straight line appears on the plot. This phenomenon only occurs sporadically and with certain data sets. I have noticed a similar phenomenon with masked arrow arrays, but that is much harder to track down. The masked elements are intended to break the plot line so that several independent polylines are plotted. (The purpose is to plot a map of coastal outlines.) I am attaching a python script which reproduces the problem, but only with a particular data set, which is also attached. Sorry, if I try to shorten the data set more than I have already, the problem goes away, even if I split the file in half an plot each half separately! I am running on a 32 bit intel processor using debian testing and the numpy and matplotlib versions are 1.3 and 0.99.1.2. However, the problem also appears on a 64 bit amd processor running debian stable with numpy and matplotlib versions 1.3 and 0.99.1.1. The python script is named maskbug.py and the data set is trunc1.dat, which is an ascii file. The data set should be read on the standard input, i. e., maskbug.py < trunc1.dat I have verified by printing the masked arrays that nothing appears to go wrong in the conversion from ascii to numpy masked array. Dave Raymond Physics Dept. New Mexico Tech Socorro, NM 87801 #!/usr/bin/python # this illustrates a bug which occurs when plotting masked data import sys from numpy import * import matplotlib.pyplot as plt # get lon-lat data from standard input and store in lists lonl = [] latl = [] while 1: inline = sys.stdin.readline() if inline == '': break else: pieces = inline.split() lonl.append(float(pieces[0])) latl.append(float(pieces[1])) # convert lists to numpy arrays lon = array(lonl) lat = array(latl) # compute masks -- masked elements have values of 1.e30 masklon = lon > 0.999e30 masklat = lat > 0.999e30 # compute masked arrays mlon = ma.array(lon, mask = masklon) mlat = ma.array(lat, mask = masklat) # list input file print mlon print mlat # plot them cc = plt.plot(mlon,mlat) ax = plt.axis([120,150,0,30]) plt.show() trunc1.dat Description: Netscape Proxy Auto Config -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] Elusive bug plotting masked arrays
On 3/19/10 11:10 AM, David J. Raymond wrote: I am trying to plot two 1-D masked arrays against each other in a line plot and an extraneous straight line appears on the plot. This phenomenon only occurs sporadically and with certain data sets. I have noticed a similar phenomenon with masked arrow arrays, but that is much harder to track down. The masked elements are intended to break the plot line so that several independent polylines are plotted. (The purpose is to plot a map of coastal outlines.) I am attaching a python script which reproduces the problem, but only with a particular data set, which is also attached. Sorry, if I try to shorten the data set more than I have already, the problem goes away, even if I split the file in half an plot each half separately! I am running on a 32 bit intel processor using debian testing and the numpy and matplotlib versions are 1.3 and 0.99.1.2. However, the problem also appears on a 64 bit amd processor running debian stable with numpy and matplotlib versions 1.3 and 0.99.1.1. The python script is named maskbug.py and the data set is trunc1.dat, which is an ascii file. The data set should be read on the standard input, i. e., maskbug.py< trunc1.dat I have verified by printing the masked arrays that nothing appears to go wrong in the conversion from ascii to numpy masked array. Dave Raymond Physics Dept. New Mexico Tech Socorro, NM 87801 Dave: Can you attach a png image showing what you get? When I run your script, I get a plot that reasonable (no obviously crazy lines running across the plot). -Jeff -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX: (303)497-6449 NOAA/OAR/PSD R/PSD1Email : jeffrey.s.whita...@noaa.gov 325 BroadwayOffice : Skaggs Research Cntr 1D-113 Boulder, CO, USA 80303-3328 Web: http://tinyurl.com/5telg -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] An easier way to create figure and group of axes; useful?
Fernando Perez wrote: > I personally think that > this should be the way to use mpl in general when scripting, and the > way I want to teach, + Inf ! I've wanted to do this for years (make a easier way to do scripting the OO way), but I only get around to a tiny fraction of the things I want to do. > For this reason, I think the name should be really > well chosen, and I'm not convinced fig_subplot is a very good one. I'll leave the name decisions to you folks, I just wanted to be encouraging! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] Elusive bug plotting masked arrays
I can't reproduce anything like a bug, either. What backend are you using? Have you tried turning path.simplify on or off? (Makes no difference here, just seems a likely candidate). Mike Jeff Whitaker wrote: > On 3/19/10 11:10 AM, David J. Raymond wrote: >> I am trying to plot two 1-D masked arrays against each other >> in a line plot and an extraneous straight line appears on >> the plot. This phenomenon only occurs sporadically and with >> certain data sets. I have noticed a similar phenomenon with >> masked arrow arrays, but that is much harder to track down. >> The masked elements are intended to break the plot line so >> that several independent polylines are plotted. (The purpose >> is to plot a map of coastal outlines.) >> >> I am attaching a python script which reproduces the problem, >> but only with a particular data set, which is also attached. >> Sorry, if I try to shorten the data set more than I have >> already, the problem goes away, even if I split the file >> in half an plot each half separately! >> >> I am running on a 32 bit intel processor using debian testing >> and the numpy and matplotlib versions are 1.3 and 0.99.1.2. >> However, the problem also appears on a 64 bit amd processor >> running debian stable with numpy and matplotlib versions >> 1.3 and 0.99.1.1. >> >> The python script is named maskbug.py and the data set is >> trunc1.dat, which is an ascii file. The data set should be >> read on the standard input, i. e., >> >> maskbug.py < trunc1.dat >> >> I have verified by printing the masked arrays that nothing >> appears to go wrong in the conversion from ascii to numpy >> masked array. >> >> Dave Raymond >> Physics Dept. >> New Mexico Tech >> Socorro, NM 87801 >> > > Dave: Can you attach a png image showing what you get? When I run > your script, I get a plot that reasonable (no obviously crazy lines > running across the plot). > > -Jeff >> >> >> -- >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> >> >> ___ >> Matplotlib-devel mailing list >> Matplotlib-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >> > > > -- > Jeffrey S. Whitaker Phone : (303)497-6313 > Meteorologist FAX: (303)497-6449 > NOAA/OAR/PSD R/PSD1Email : jeffrey.s.whita...@noaa.gov > 325 BroadwayOffice : Skaggs Research Cntr 1D-113 > Boulder, CO, USA 80303-3328 Web: http://tinyurl.com/5telg > > > > -- > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > > > ___ > Matplotlib-devel mailing list > Matplotlib-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
[matplotlib-devel] mplot3d view_init minor patch
Here is a minor patch for the mplot3d.view_init() function. It:
1. comments this method so that users know that they can use it to
programatically rotate or re-set the axes.
2. adds some new functionality so that if no parameters are passed in, the
default values (what was specified in the Axes3D constructor) are used instead.
This diff was created with SVN head revision 8199.
I tested the new feature with my own code to verify it works, but don't have
any example code or test scripts to submit.
Thanks,
-BenIndex: lib/mpl_toolkits/mplot3d/axes3d.py
===
--- lib/mpl_toolkits/mplot3d/axes3d.py (revision 8199)
+++ lib/mpl_toolkits/mplot3d/axes3d.py (working copy)
@@ -59,8 +59,8 @@
self.fig = fig
self._cids = []
-azim = kwargs.pop('azim', -60)
-elev = kwargs.pop('elev', 30)
+self.initial_azim = kwargs.pop('azim', -60)
+self.initial_elev = kwargs.pop('elev', 30)
self.xy_viewLim = unit_bbox()
self.zz_viewLim = unit_bbox()
@@ -68,7 +68,7 @@
self.zz_dataLim = unit_bbox()
# inihibit autoscale_view until the axises are defined
# they can't be defined until Axes.__init__ has been called
-self.view_init(elev, azim)
+self.view_init(self.initial_elev, self.initial_azim)
self._ready = 0
Axes.__init__(self, self.fig, rect,
frameon=True,
@@ -272,11 +272,30 @@
def panpy(self, numsteps):
print 'numsteps', numsteps
-def view_init(self, elev, azim):
+def view_init(self, elev=None, azim=None):
+"""Set the elevation and azimuth of the axes.
+
+This can be used to rotate the axes programatically.
+
+'elev' stores the elevation angle in the z plane.
+'azim' stores the azimuth angle in the x,y plane.
+
+if elev or azim are None (default), then the initial value
+is used which was specified in the :class:`Axes3D` constructor.
+"""
+
self.dist = 10
-self.elev = elev
-self.azim = azim
-
+
+if elev is None:
+self.elev = self.initial_elev
+else:
+self.elev = elev
+
+if azim is None:
+self.azim = self.initial_azim
+else:
+self.azim = azim
+
def get_proj(self):
"""Create the projection matrix from the current viewing
position.
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] Elusive bug plotting masked arrays
David J. Raymond wrote: > I am using python 2.5.5 and the gtk background (as far as I can tell). > Turning off path.simplify gets rid of the extraneous line. I am > attaching pngs with path.simplify both on and off. I am also > attaching the full coastline file that produced the original problem. > I noticed that the behavior was a bit weird, with the bad line > appearing or disappearing in apparently random ways depending on what > was edited out of the input file. This looks to me like a bug that Mike fixed, maybe the one where a path was not always starting with a move_to. That was fixed on October 22. > > By the way, I notice that the default value of > 'path.simplify_threshold' is 0.... I don't quite know what this > means, but it sounds pertinent. > > I assume path.simplify is some kind of optimization scheme? Yes, it is a very fancy and nice set of algorithms for eliminating path elements that would not make any visible difference to the output. It makes it possible to plot very large datasets. Eric > > Dave > > On Fri, Mar 19, 2010 at 03:13:04PM -0400, Michael Droettboom wrote: >> I can't reproduce anything like a bug, either. What backend are you >> using? Have you tried turning path.simplify on or off? (Makes no >> difference here, just seems a likely candidate). >> >> Mike >> >> Jeff Whitaker wrote: >>> On 3/19/10 11:10 AM, David J. Raymond wrote: I am trying to plot two 1-D masked arrays against each other in a line plot and an extraneous straight line appears on the plot. This phenomenon only occurs sporadically and with certain data sets. I have noticed a similar phenomenon with masked arrow arrays, but that is much harder to track down. The masked elements are intended to break the plot line so that several independent polylines are plotted. (The purpose is to plot a map of coastal outlines.) I am attaching a python script which reproduces the problem, but only with a particular data set, which is also attached. Sorry, if I try to shorten the data set more than I have already, the problem goes away, even if I split the file in half an plot each half separately! I am running on a 32 bit intel processor using debian testing and the numpy and matplotlib versions are 1.3 and 0.99.1.2. However, the problem also appears on a 64 bit amd processor running debian stable with numpy and matplotlib versions 1.3 and 0.99.1.1. The python script is named maskbug.py and the data set is trunc1.dat, which is an ascii file. The data set should be read on the standard input, i. e., maskbug.py < trunc1.dat I have verified by printing the masked arrays that nothing appears to go wrong in the conversion from ascii to numpy masked array. Dave Raymond Physics Dept. New Mexico Tech Socorro, NM 87801 >>> Dave: Can you attach a png image showing what you get? When I >>> run your script, I get a plot that reasonable (no obviously crazy >>> lines running across the plot). >>> >>> -Jeff -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >>> >>> -- >>> Jeffrey S. Whitaker Phone : (303)497-6313 >>> Meteorologist FAX: (303)497-6449 >>> NOAA/OAR/PSD R/PSD1Email : jeffrey.s.whita...@noaa.gov >>> 325 BroadwayOffice : Skaggs Research Cntr 1D-113 >>> Boulder, CO, USA 80303-3328 Web: http://tinyurl.com/5telg >>> >>> >>> -- >>> Download Intel® Parallel Studio Eval >>> Try the new software tools for yourself. Speed compiling, find bugs >>> proactively, and fine-tune applications for parallel performance. >>> See why Intel Parallel Studio got high marks during beta. >>> http://p.sf.net/sfu/intel-sw-dev >>> >>> >>> ___ >>> Matplotlib-devel mailing list >>> Matplotlib-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel >> -- >> Michael Droettboom >> Science Software Branch >> Operations and Engineering Division >> Space Telescope Science Institute >> Operated by AURA for NASA >> >> >> ---
