[Matplotlib-users] Is it possible to build matplotlib in place ?
Hi,
I tried to build matplotlib in place (setup.py build_ext -i), and
found out that I could not import it:
matplotlib/rcsetup.py:117: UserWarning: rcParams key "numerix" is
obsolete and has no effect;
please delete it from your matplotlibrc file
warnings.warn('rcParams key "numerix" is obsolete and has no effect;\n'
Traceback (most recent call last):
File "", line 1, in
File "pylab.py", line 1, in
from matplotlib.pylab import *
File "matplotlib/pylab.py", line 206, in
from matplotlib import mpl # pulls in most modules
File "matplotlib/mpl.py", line 2, in
from matplotlib import axis
File "matplotlib/axis.py", line 10, in
import matplotlib.font_manager as font_manager
File "matplotlib/font_manager.py", line 52, in
from matplotlib import ft2font
ImportError: matplotlib/ft2font.so: undefined symbol: py_object_initializer
Is this expected ?
cheers,
David
--
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Is it possible to build matplotlib in place ?
Hi Sandro,
(sorry for the private reply)
Sandro Tosi wrote:
> Hi David,
>
> On Wed, Sep 30, 2009 at 07:48, David Cournapeau
> wrote:
>
>> Hi,
>>
>>I tried to build matplotlib in place (setup.py build_ext -i), and
>> found out that I could not import it:
>>
>
> yes, you should be able to build it in-place (even if usually run
> 'python setup.py build' because I need the full package be available
> in build/ ) even from the unpacked tarball or from svn
> trunk.
>
>
>> matplotlib/rcsetup.py:117: UserWarning: rcParams key "numerix" is
>> obsolete and has no effect;
>> please delete it from your matplotlibrc file
>> warnings.warn('rcParams key "numerix" is obsolete and has no effect;\n'
>> Traceback (most recent call last):
>> File "", line 1, in
>> File "pylab.py", line 1, in
>>from matplotlib.pylab import *
>> File "matplotlib/pylab.py", line 206, in
>>from matplotlib import mpl # pulls in most modules
>> File "matplotlib/mpl.py", line 2, in
>>from matplotlib import axis
>> File "matplotlib/axis.py", line 10, in
>>import matplotlib.font_manager as font_manager
>> File "matplotlib/font_manager.py", line 52, in
>>from matplotlib import ft2font
>> ImportError: matplotlib/ft2font.so: undefined symbol: py_object_initializer
>>
>
> what version are you using? 0.99.1.1, svn trunk, other?
>
It ended up a confusion between the distutils build and the new scons
scripts I was working on, sorry for the noise.
The good news is that matplotlib can now be built with numscons, with
all scons goodies :)
David
--
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Is it possible to build matplotlib in place ?
Michael Droettboom wrote: > David Cournapeau wrote: >> >> It ended up a confusion between the distutils build and the new scons >> scripts I was working on, sorry for the noise. >> >> The good news is that matplotlib can now be built with numscons, with >> all scons goodies :) >> > Please share! :) That's my intention, but github downtimes and anal firewall rules at my university makes this challenging at the moment, unfortunately David -- Come build with us! The BlackBerry® Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9-12, 2009. Register now! http://p.sf.net/sfu/devconf ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] matplotlib setup is trying to build macosx extension on linux ?
Hi, I updated matplotlib to 0.99.1, and got a weird build failure: gcc: error trying to exec 'cc1obj': execvp: No such file or directory I realized that cc1obj stands for the objective C compiler, and that it is trying to build the macosx extension, but I am on Ubuntu with python 2.6, so it does not make much sense. The following lines in setup.py are a bit suspicious: if options['build_macosx']: if check_for_macosx() or (options['build_macosx'] is True): build_macosx(ext_modules, packages) rc['backend'] = 'MacOSX' This would always build the mac os x extension if options['build_macosx'] is True, which is the case for me (it is set-up in setup.cfg, which I did not touch). I am a bit surprised to see this, as I am sure I am far from being the first one to build this version of matplotlib on Ubuntu, cheers, David -- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] matplotlib setup is trying to build macosx extension on linux ?
Jouni K. Seppänen wrote: > David Cournapeau writes: > > > The setup.cfg file is included in the distribution by mistake. Just > delete it before building. > Ah, that would explain it, indeed. Thanks, David -- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] C compiler is being used to compile C++ code.
Dr. David Kirkby wrote: > Well, on SPARC, the Sun compilers are better than gcc, so there really is an > advantage to getting this sorted out. > numscons should support the Sun Studio compiler. > I do not know SCons myself, but I do know that in the Sage project, SCons has > caused many issues. Whether it is a fault of SCons, or the way people write > the > code for SCons, I do not know. > That's because scons is a terrible tool for a full packaging solution: at the moment, it is really just a (quite good) make replacement. But numscons does not have this problem, as it reuses distutils for all the packaging, and handles compilers in its own way. I am pretty sure it will be easier to fix whatever numscons issues crop up while trying to build matplotlib than building it from scratch or worse, trying to fix distutils. cheers, David -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] How to save figures in black and white
Hi, I am using matplotlib for all my figure needs in academic papers, but I would like to convert some of my figures (generally exported ni eps and pdf) in black and white. I tried using colormaps, but when using several axes, I can't convert them all in one shot. Is there any way to say (in say savefig) to use a given colormap (gray levels in my case) for the whole figure ? Or is colormap not the way to go ? thanks, David - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Python 2.6
John Hunter wrote: > On Thu, Nov 6, 2008 at 4:28 PM, Darren Dale <[EMAIL PROTECTED]> wrote: > > >> Stan West checked out my subprocess patch on windows with python-2.5, which >> should take care of a bunch of deprecation warnings. I need to double check >> that I got them all, maybe I can get to it this weekend. >> >> I'm in favor of dropping support for python-2.4, but on the other hand I >> think the most recent version of RHEL still uses this version. >> > > Actually, we still use 2.4 at work, so I'd like to continue supporting > 2.4 for a while I guess, for purely selfish reasons. But perhaps we > should stop making binaries for it to ease the burden on Charlie. > Once the 2.6 binaries for numpy are out and we are making binaries for > the next release, that is > I think it would be a mistake to stop supporting python 2.4 as well. RHEL indeed still uses 2.4 as its default python. It would make the installation of the numpy/mpl stack even harder than it already is on those platforms, which does not strike me as a good idea (I am a numpy developer, and I find it already quite difficult). Does python 2.5 have that many interesting features compared to 2.4 ? cheers, David - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Matplotlib and Python 3
Kaushik Ghose wrote: > Hi Everyone, > > Quick question. Is matplotlib python 3 compatible? Has anyone switched to > python > 3? Anecdotally, how much of a pain is it to switch over, if you use common > scientific libraries such as PIL and VTK? > matplotlib depends on numpy, and numpy on python 3 won't be finished anytime soon - it will be a huge task. David -- SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Pylab breaks locales with GTK
Hi Ryan, Ryan May wrote: > Lionel Roubeyrie wrote: > >> Hi all, >> Trying to write to text files some plotted datas, we have a strange >> behavour on masked arrays after importing pylab, with the dot decimal >> separator replaced by a comma (but not all) : >> ## >> Python 2.5.2 (r252:60911, Oct 5 2008, 19:24:49) >> [GCC 4.3.2] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> > > This has been reported before, and IIRC, it's a problem with PyGTK. > The fact that pygtk changes the local at import sounds buggy indeed, but numpy should not be dependent on the locale anyway, so it also shows a numpy bug I think: http://scipy.org/scipy/numpy/ticket/884 Pauli and other have been working on those locale-related bugs in numpy, but they are a bit fastidious to fix - hopefully, they will make their way into numpy 1.3 David -- ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] [Numpy-discussion] Anybody going to PyCon?
On Mon, Mar 7, 2011 at 5:53 PM, Gökhan Sever wrote: > Hello, > > I am going to the PyCon this week. I am presenting a poster about an > atmospheric sciences related project -- the most active development > from my coding site over at http://code.google.com/p/ccnworks/ > > Is there anybody in the community participating there as well? Any > plans for sprinting or similar activities? Will be there as well, but only for the WE + monday, cheers, David -- Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
David Cournapeau wrote: > Eric Firing wrote: >> There is a clip function in all three numeric packages, so a native >> clip is being used. >> >> If numpy.clip is actually slower than your version, that sounds like a >> problem with the implementation in numpy. By all logic a single clip >> function should either be the same (if it is implemented like yours) >> or faster (if it is a single loop in C-code, as I would expect). This >> warrants a little more investigation before changing the mpl code. >> The best thing would be if you could make a simple standalone numpy >> test case profiling both versions and post the results as a question >> to the numpy-discussion list. Many such questions in the past have >> resulted in big speedups in numpy. > I am much more familiar with internal numpy code than matplotlib's, so > this is much easier for me, too :) >> One more thought: it is possible that the difference is because myclip >> operates on the array in place while clip generates a new array. If >> this is the cause of the difference then changing your last line to >> "return a.copy()" probably would slow it down to the numpy clip speed >> or slower. > It would be scary if a copy of a 8008x256 array of double took 100 ms... > Fortunately, it does not, this does not seem to be the problem. > > cheers, > > David Ok, so now, with my clip function, still for a 8000x256 double array: we have show() after imshow which takes around 760 ms. 3/5 are in make_image, 2/5 in the function blop, which is just an alias I put to measure the difference between axes.py:1043(draw) and image.py:173(draw) in the function Axis.draw (file axes.py): def blop(dsu): for zorder, i, a in dsu: a.draw(renderer) blop(dsu) In make_image, most of the time is taken into to_rgba: almost half of it is taken in by the take call in the Colormap.__call__. Almost 200 ms to get colors from the indexes seems quite a lot (this means 280 cycles / pixel on average !). I can reproduce this number by using a small numpy test. On my laptop (pentium M, 1.2 Ghz), make_image takes almost 85 % of the time, which seems to imply that this is where one should focus if one wants to improve the speed, cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
John Hunter wrote: > > > David> In make_image, most of the time is taken into to_rgba: > David> almost half of it is taken in by the take call in the > David> Colormap.__call__. Almost 200 ms to get colors from the > David> indexes seems quite a lot (this means 280 cycles / pixel on > David> average !). I can reproduce this number by using a small > David> numpy test. > > David> On my laptop (pentium M, 1.2 Ghz), make_image takes almost > David> 85 % of the time, which seems to imply that this is where > David> one should focus if one wants to improve the speed, > > This may have been lost in the longer thread above, I am a bit lost myself between numpy and mpl ML, sorry for the inconvenience. > but what > interpolation are you using? You may see a good performance boost by > using interpolation='nearest'. At what point is interpolation used ? > Also, with your clip changes and with > Eric's changes is it still painfully slow for you Painfully is a strong word :) It is still 10 to 15 times slower than matlab on the same computer: the show call is around 800 ms instead of 70 ms with matlab, and matlab image is equivalent to imshow + show calls actually. Matlab having only one toolkit, it obviously has an advantage, but I don't think the problem is on the GUI side anyway. > -- how much have > these changes helped? With the original profiling, it took a bit more than 2100 ms for a show call after a imshow call for a 8000x256 array according to a saved kcachegrind profile. Now, it is around 800 ms, which is already much better, and with minimal changes (eg without using a special fast path more prone to bugs). I estimate that squeezing to a bit less than 500 ms should be easily possible by improving on numpy side (clip, float to int convertion and take function), which has the nice effect of improving mpl without touching one line of it, and improving numpy as the same time :) The last 500 ms would be much more difficult to squeeze: half of it is used to 'launch' the figure anyway. And below a few hundred ms, it is becoming unnoticeable in interactive use (whereas the change from 2.1 s to 0.8 is; on my laptop, it is even more noticeable, because its CPU is kind of slow). David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] Numpy 1.0.2: some improvements for matplotlib
Hi there, A few months back, I complained about the slowness of the image function in matplotlib. One of the cullprit was a slow clip function; I've done a bit some work to improve the situation on numpy's side, efforts which were integrated in numpy 1.0.2. Now, when you clip a numpy array with scalar min and max values, you get a 5 to 30 fold speed-up; to get the maximum efficiency, you need inplace clipping (using the syntax a.clip(min, max, a) for a a numpy array). This makes image significantly faster (between 100 and 200 ms on recent computers), and I am sure in other functionalities of matplotlib as well. cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] Thightening the bounding box while exporting ?
Hi, Is there a simple way to tighten the bounding box of a figure while exporting to eg png or pdf ? For now, I am modifying the bounding box directly in the files, which is far from optimal. I would like to get less space between the bounding box and the axis when using subplot. cheers, David - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] etframes: Applying the ideas of Edward Tufte to matplotlib
Andrew Straw wrote: > I came across this piece by Adam Hupp on programming.reddit.com just > now. It looks interesting: > > http://hupp.org/adam/weblog/2007/09/03/etframes-applying-the-ideas-of-edward-tufte-to-matplotlib/ > > > This is indeed really interesting. Thanks for this link, David - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
Hi, I am a regular user of matplotlib since I moved from matlab to python/numpy/scipy. Even if I find matplotlib to be a real help during the transition from matlab to python, I must confess I found it the most disappointing compare other packages ( essentially numpy/scipy/ipython). This is not a rant; I want to know if this slowness is coming from my lack of matplotlib knowledge or not; I apologize in advance if the following hurts anyone feelings :) First, I must admit that whereas I took a significant amount of time to study numpy and scipy, I didn't take that same time for matplotlib. So this disappointment may just be a consequences of this laziness. My main problem with matplotlib is speed: I find it really annoying to use in an interactive manner. For example, when I need to display some 2d information, such as spectrogramm or correlogram, this take 1 or 2 seconds for a small signal (~4500 frames of 256 samples). My function correlogram (similar to specgram, but compute correlation instead of log spectrum) uses imshow, and this function takes 20 times more time than imagesc of matlab for the same size. Also, I found changing the size of the matplotlib window really 'annoying to the eye': I compared to matlab, and this may be due to the fact that the whole window is redrawn with matplotlib, including the toolbar, whereas in matlab, the top toolbar is not redrawn. Finally, plotting many data (using plot(X, Y) with X and Y around 1000/1 samples) is 'slow' (the '' are because I don't know much about computer graphics, and I understand that slow in the rendering is often just a perception) So, is this a current limitation of matplotlib, is matplotlib optimized for good rendering for publication, and not for interactive use, or I am just misguided in my use of matplotlib ? Config info: - ubuntu edgy on a bi xeon 3.2 Ghz with 2 Gb of Ram - numpy SVN (post 1.0) - matplotlib 0.87.7 - matplotlibrc: uses numpy for numeric, Gtk as a backend (or GtkAdd for anti aliasing, but this makes the problem worse). Cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
John Hunter wrote: This is where you can help us. Saying specgram is slow is only marginally more useful than saying matplotlib is slow or python is slow. What is helpful is to post a complete, free-standing script that we can run, with some attached performance numbers. For starters, just run it with the Agg backend so we can isolate matplotlib from the respective GUIs. Show us how the performance scales with the specgram parameters (frames and samples). specgram is divided into two parts (if you look at the Axes.specgram you will see that it calls matplotlib.mlab.specgram to do the computation and Axes.imshow to visualize it. Which part is slow: the mlab.specgram computation or the visualizion (imshow) part or both? You can paste this function into your own python file and start timing different parts. The most helpful "this is slow" posts come with profiler output so we can see where the bottlenecks are. (sorry for double posting) Ok, here we go: I believe that the rendering of the figure returned by imshow to be slow. For example, let's say I have a 2 minutes signal @ 8kHz sampling-rate, with windows of 256 samples with 50 % overlap. I have around 64 frames / seconds, eg ~ 8000 frames of 256 windows. So for benchmark purposes, we can just send random data of shape 8000x256 to imshow. In ipython, this takes a long time (around 2 seconds for imshow(data), where data = random(8000, 256)). Now, on a small script to have a better idea: import numpy as N import pylab as P def generate_data_2d(fr, nwin, hop, len): nframes = 1.0 * fr / hop * len return N.random.randn(nframes, nwin) def bench_imshow(fr, nwin, hop, len, show = True): data= generate_data_2d(fr, nwin, hop, len) P.imshow(data) if show: P.show() if __name__ == '__main__': # 2 minutes (120 sec) of sounds @ 8 kHz with 256 samples with 50 % overlap bench_imshow(8000, 256, 128, 120, show = False) Now, I have a problem, because I don't know how to benchmark when using show to True (I have to manually close the figure). If I run the above script with time, I got 1.5 seconds with show = False (after several trials to be sure matplotlib files are in the system cache: this matters because my home dir is on NFS). If I set show = True, and close the figure by hand once the figure is plotted, I have 4.5 sec instead. If I run the above script with -dAgg --versbose-helpful (I was looking for this one to check numerix is correctly set to numpy:) ): with show = False: matplotlib data path /home/david/local/lib/python2.4/site-packages/matplotlib/mpl-data $HOME=/home/david CONFIGDIR=/home/david/.matplotlib loaded rc file /home/david/.matplotlib/matplotlibrc matplotlib version 0.87.7 verbose.level helpful interactive is False platform is linux2 numerix numpy 1.0.2.dev3484 font search path ['/home/david/local/lib/python2.4/site-packages/matplotlib/mpl-data'] loaded ttfcache file /home/david/.matplotlib/ttffont.cache backend Agg version v2.2 real0m1.185s user0m0.808s sys 0m0.224s with show = True matplotlib data path /home/david/local/lib/python2.4/site-packages/matplotlib/mpl-data $HOME=/home/david CONFIGDIR=/home/david/.matplotlib loaded rc file /home/david/.matplotlib/matplotlibrc matplotlib version 0.87.7 verbose.level helpful interactive is False platform is linux2 numerix numpy 1.0.2.dev3484 font search path ['/home/david/local/lib/python2.4/site-packages/matplotlib/mpl-data'] loaded ttfcache file /home/david/.matplotlib/ttffont.cache backend Agg version v2.2 real0m1.193s user0m0.848s sys 0m0.192s So the problem is in the rendering, right ? (Not sure to understand exactly what Agg backend is doing). Now, using hotshot (kcache grind profiles attached to the email), for the noshow case: 10.0010.0010.8390.839 slowmatplotlib.py:181(bench_imshow_noshow) 10.0000.0000.8370.837 slowmatplotlib.py:163(bench_imshow) 10.0000.0000.5860.586 /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:1894(imshow) 30.0000.0000.5100.170 /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:883(gca) 10.0000.0000.5090.509 /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:950(ishold) 40.0000.0000.4090.102 /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:903(gcf) 10.0000.0000.4090.409 /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:818(figure) 10.0000.0000.4080.408 /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtkagg.py:36(new_figure_manager) 10.0030.0030.4000.400 /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py:
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
David Cournapeau wrote: > But the show case is more interesting: > > ncalls tottime percall cumtime percall filename:lineno(function) > 10.0020.0023.8863.886 > slowmatplotlib.py:177(bench_imshow_show) > 10.0000.0003.8843.884 > slowmatplotlib.py:163(bench_imshow) > 10.6980.6983.0033.003 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py:70(show) > > > 20.0000.0002.2661.133 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py:275(expose_event) > > > 10.0090.0092.2662.266 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtkagg.py:71(_render_figure) > > > 10.0000.0002.2562.256 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_agg.py:385(draw) > > > 10.0000.0002.2532.253 > /home/david/local/lib/python2.4/site-packages/matplotlib/figure.py:510(draw) > > 10.0000.0002.2512.251 > /home/david/local/lib/python2.4/site-packages/matplotlib/axes.py:994(draw) > > 10.0050.0051.9511.951 > /home/david/local/lib/python2.4/site-packages/matplotlib/image.py:173(draw) > > 10.0960.0961.9461.946 > /home/david/local/lib/python2.4/site-packages/matplotlib/image.py:109(make_image) > > > 10.0020.0021.8501.850 > /home/david/local/lib/python2.4/site-packages/matplotlib/cm.py:50(to_rgba) > > 10.0010.0010.9490.949 > /home/david/local/lib/python2.4/site-packages/matplotlib/colors.py:735(__call__) > > > 10.0970.0970.8990.899 > /home/david/local/lib/python2.4/site-packages/matplotlib/colors.py:568(__call__) > > > 3250.0500.0000.6710.002 > /home/david/local/lib/python2.4/site-packages/numpy/core/ma.py:533(__init__) > > 10.6000.6000.6000.600 > /home/david/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py:282(resize) > > > 10.0000.0000.5960.596 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:1894(imshow) > > > 100.5700.0570.5700.057 > /home/david/local/lib/python2.4/site-packages/numpy/oldnumeric/functions.py:117(where) > > > 30.0000.0000.5130.171 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:883(gca) > > 10.0000.0000.5130.513 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:950(ishold) > > 40.0000.0000.4080.102 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:903(gcf) > > > For more details, see the .kc files which are the in the tbz2 archive, > with the script for generating profiles for kcachegrind, Here is some stuff I tried: - first, we can see that in expose_event (one is expensive, the other negligeable, from my understanding), two calls are pretty expensive: the __call__ at line 735 (for normalize functor) and one for __call__ at line 568 (for colormap functor). - for normalize functor, one line is expensive: val = ma.array(clip(val.filled(vmax), vmin, vmax), mask=mask). If I put a test on mask when mask is None (which it is in my case), then the function becomes negligeable. - for colormap functor, the 3 where calls are expensive. I am not sure to understand in which case they are useful; if I understand correctly, one tries to avoid values out of range (0, N), and force out of range values to be clipped. Isn't there an easier way than using where ? If I remove the where in the colormap functor, I have a 4x speed increase for the to_rgba function. After that, it becomes a bit more tricky to change things for someone like me who have no knowledge about matplotlib internals. Cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
Eric Firing wrote: > > Regarding the clip line, I think that your test for mask is None is > not the right solution because it knocks out the clipping operation, > but the clipping is intended regardless of the state of the mask. I > had expected it to be a very fast operation, so I am surprised it is a > bottleneck; in any case I can take a look to see how it can be sped > up, or whether it can be bypassed in some cases. Maybe it is also > using "where" internally. (again, sorry for the double posting, I always forget that some ML do not reply automatically to the ML) My wordings were vague at best :) The clipping operation is *not* removed, and it was not the culprit (it becomes a bottleneck once you get the 4x speed issue, though). What I did was: if self.clip: mask = ma.getmaskorNone(val) if mask == None: val = ma.array(clip(val.filled(vmax), vmin, vmax)) else: val = ma.array(clip(val.filled(vmax), vmin, vmax), mask=mask) Actually, the problem is in ma.array: with a value of mask to None, it should not make a difference between mask = None or no mask arg, right ? I didn't change ma.array to keep my change as local as possible. To change only this operation as above gives a speed up from 1.8 s to ~ 1.0 s for to_rgba, which means calling show goes from ~ 2.2 s to ~1.4 s. I also changed result = (val-vmin)/float(vmax-vmin) to invcache= 1.0 / (vmax - vmin) result = (val-vmin) * invcache which gives a moderate speed up (around 100 ms for a 8000x256 points array, still in the 5-10 % range of the whole cost, though, and is not likely to cause any hidden bug). Once you make both those changes, the clip call is by far the most expensive operation in normalize functor, but the functor is not really expensive anymore compared to the rest, so this is not where I looked at after. For the where calls in Colormap functor, I was wondering if they are necessary in all cases: some of those calls seem redundant, and it may be possible to detect that before calling them. This should be both easier and faster, at least in this case, than having a fast where ? I understand that support of multiple array backend, support of mask arrays have cost consequences. But it looks like it may be possible to speed things up for cases where an array has only meaningful values/no mask. cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
Eric Firing wrote: > David, > > I have made some changes in svn that address all but one of the points > you made: > > [] >> if self.clip: >>mask = ma.getmaskorNone(val) >>if mask == None: >>val = ma.array(clip(val.filled(vmax), vmin, vmax)) >>else: >>val = ma.array(clip(val.filled(vmax), vmin, vmax), >>mask=mask) > > The real problem here is that I should not have been using > getmaskorNone(). In numpy.ma, we need nomask, not None, so we want an > ordinary getmask() call. ma.array(, mask=ma.nomask) is very fast, > so the problem goes away. > >> >> Actually, the problem is in ma.array: with a value of mask to None, >> it should not make a difference between mask = None or no mask arg, >> right ? > But it does, because for numpy it needs to be nomask; it does > something with None, but whatever it is, it is very slow. > >> I didn't change ma.array to keep my change as local as possible. To >> change only this operation as above gives a speed up from 1.8 s to ~ >> 1.0 s for to_rgba, which means calling show goes from ~ 2.2 s to ~1.4 >> s. I also changed result = (val-vmin)/float(vmax-vmin) >> >> to >> >> invcache= 1.0 / (vmax - vmin) >> result = (val-vmin) * invcache > > This is the one I did not address. I don't understand how this could > be making much difference, and some testing using ipython and %prun > with 1-line operations showed little difference with variations on > this theme. The fastest would appear to be (and logically should be, > I think) result = (val-vmin)*(1.0/(vmax-vmin)), but I don't think it > makes much difference--it looks to me like maybe 10-20 msec, not 100, > on my Pentium M 1.6 Ghz. Maybe still worthwhile, so I may yet make > the change after more careful testing. > > >> >> which gives a moderate speed up (around 100 ms for a 8000x256 points >> array). Once you make both those changes, the clip call is by far the >> most expensive operation in normalize functor, but the functor is not >> really expensive anymore compared to the rest, so this is not where I >> looked at. >> >> For the where calls in Colormap functor, I was wondering if they are >> necessary in all cases: some of those calls seem redundant, and it >> may be possible to detect that before calling them. This should be >> both easier and faster, at least in this case, than having a fast >> where ? >> > > You hit the nail squarely: where() is the wrong function to use, and I > have eliminated it from colors.py. The much faster replacement is > putmask, which does as well as direct indexing with a Boolean but > works with all three numerical packages. I think that using the fast > putmask is better than trying to figure out special cases in which > there would be nothing to put, although I could be convinced otherwise. > > >> I understand that support of multiple array backend, support of mask >> arrays have cost consequences. But it looks like it may be possible >> to speed things up for cases where an array has only meaningful >> values/no mask. > > The big gains here were essentially bug fixes--picking the appropriate > function (getmask versus getmaskorNone and putmask versus where). Ok, I've installed last svn, and now, there is still one function which is much slower than a direct numpy implementation, so I would like to know if this is inherent to the multiple backend nature of matplotlib or not. The functor Normalize uses the clip function, and a direct numpy would be 3 times faster (giving the show call a 20 % speed in my really limited benchmarks): if clip: mask = ma.getmask(val) #val = ma.array(nx.clip(val.filled(vmax), vmin, vmax), #mask=mask) def myclip(a, m, M): a[aM] = M return a val = ma.array(myclip(val.filled(vmax), vmin, vmax), mask=mask) I am a bit lost in the matplotlib code to see where clip is implemented (is it in numerix and as such using the numpy function clip ?). Still, I must confess that all this looks quite good, because it was possible to speed things up quite considerably without too much effort, cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
Eric Firing wrote: > > There is a clip function in all three numeric packages, so a native > clip is being used. > > If numpy.clip is actually slower than your version, that sounds like a > problem with the implementation in numpy. By all logic a single clip > function should either be the same (if it is implemented like yours) > or faster (if it is a single loop in C-code, as I would expect). This > warrants a little more investigation before changing the mpl code. > The best thing would be if you could make a simple standalone numpy > test case profiling both versions and post the results as a question > to the numpy-discussion list. Many such questions in the past have > resulted in big speedups in numpy. I am much more familiar with internal numpy code than matplotlib's, so this is much easier for me, too :) > > One more thought: it is possible that the difference is because myclip > operates on the array in place while clip generates a new array. If > this is the cause of the difference then changing your last line to > "return a.copy()" probably would slow it down to the numpy clip speed > or slower. It would be scary if a copy of a 8008x256 array of double took 100 ms... Fortunately, it does not, this does not seem to be the problem. cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
