Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
Eric Firing wrote: > > Regarding the clip line, I think that your test for mask is None is > not the right solution because it knocks out the clipping operation, > but the clipping is intended regardless of the state of the mask. I > had expected it to be a very fast operation, so I am surprised it is a > bottleneck; in any case I can take a look to see how it can be sped > up, or whether it can be bypassed in some cases. Maybe it is also > using "where" internally. (again, sorry for the double posting, I always forget that some ML do not reply automatically to the ML) My wordings were vague at best :) The clipping operation is *not* removed, and it was not the culprit (it becomes a bottleneck once you get the 4x speed issue, though). What I did was: if self.clip: mask = ma.getmaskorNone(val) if mask == None: val = ma.array(clip(val.filled(vmax), vmin, vmax)) else: val = ma.array(clip(val.filled(vmax), vmin, vmax), mask=mask) Actually, the problem is in ma.array: with a value of mask to None, it should not make a difference between mask = None or no mask arg, right ? I didn't change ma.array to keep my change as local as possible. To change only this operation as above gives a speed up from 1.8 s to ~ 1.0 s for to_rgba, which means calling show goes from ~ 2.2 s to ~1.4 s. I also changed result = (val-vmin)/float(vmax-vmin) to invcache= 1.0 / (vmax - vmin) result = (val-vmin) * invcache which gives a moderate speed up (around 100 ms for a 8000x256 points array, still in the 5-10 % range of the whole cost, though, and is not likely to cause any hidden bug). Once you make both those changes, the clip call is by far the most expensive operation in normalize functor, but the functor is not really expensive anymore compared to the rest, so this is not where I looked at after. For the where calls in Colormap functor, I was wondering if they are necessary in all cases: some of those calls seem redundant, and it may be possible to detect that before calling them. This should be both easier and faster, at least in this case, than having a fast where ? I understand that support of multiple array backend, support of mask arrays have cost consequences. But it looks like it may be possible to speed things up for cases where an array has only meaningful values/no mask. cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] 'fill' command broken in svn
It's fixed now. Eric Rob Hetland wrote: > fill(x, y) returns an error like: [] > /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- > packages/matplotlib/axes.py in update_datalim(self, xys) > 966 # and the data in xydata > 967 xys = asarray(xys) > --> 968 self.dataLim.update_numerix_xy(xys, -1) > 969 > 970 > > : Bbox::update_numerix_xy expected > numerix array - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] 'fill' command broken in svn
Rob, OK, thanks. That sounds like something resulting from the change I made to support 2D array input to plot. I will check it. Eric Rob Hetland wrote: > fill(x, y) returns an error like: > > > /Users/rob/Projects/Merrimack/Grid/landfill.py in () > 24 for filename in filenames: > 25 x, y, = pl.load(filename).T > ---> 26 pl.fill(x, y, facecolor=fillcolor, alpha=fillalpha) > 27 > 28 > > /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- > packages/matplotlib/pylab.py in fill(*args, **kwargs) > 1869 hold(h) > 1870 try: > -> 1871 ret = gca().fill(*args, **kwargs) > 1872 draw_if_interactive() > 1873 except: > > /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- > packages/matplotlib/axes.py in fill(self, *args, **kwargs) > 3677 patches = [] > 3678 for poly in self._get_patches_for_fill(*args, > **kwargs): > -> 3679 self.add_patch( poly ) > 3680 patches.append( poly ) > 3681 self.autoscale_view() > > /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- > packages/matplotlib/axes.py in add_patch(self, p) > 951 xys = self._get_verts_in_data_coords( > 952 p.get_transform(), p.get_verts()) > --> 953 self.update_datalim(xys) > 954 self.patches.append(p) > 955 > > /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- > packages/matplotlib/axes.py in update_datalim(self, xys) > 966 # and the data in xydata > 967 xys = asarray(xys) > --> 968 self.dataLim.update_numerix_xy(xys, -1) > 969 > 970 > > : Bbox::update_numerix_xy expected > numerix array > WARNING: Failure executing file: > > > > Rob Hetland, Associate Professor > Dept. of Oceanography, Texas A&M University > http://pong.tamu.edu/~rob > phone: 979-458-0096, fax: 979-845-6331 > > > > - > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > ___ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] 'fill' command broken in svn
fill(x, y) returns an error like: /Users/rob/Projects/Merrimack/Grid/landfill.py in () 24 for filename in filenames: 25 x, y, = pl.load(filename).T ---> 26 pl.fill(x, y, facecolor=fillcolor, alpha=fillalpha) 27 28 /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- packages/matplotlib/pylab.py in fill(*args, **kwargs) 1869 hold(h) 1870 try: -> 1871 ret = gca().fill(*args, **kwargs) 1872 draw_if_interactive() 1873 except: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- packages/matplotlib/axes.py in fill(self, *args, **kwargs) 3677 patches = [] 3678 for poly in self._get_patches_for_fill(*args, **kwargs): -> 3679 self.add_patch( poly ) 3680 patches.append( poly ) 3681 self.autoscale_view() /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- packages/matplotlib/axes.py in add_patch(self, p) 951 xys = self._get_verts_in_data_coords( 952 p.get_transform(), p.get_verts()) --> 953 self.update_datalim(xys) 954 self.patches.append(p) 955 /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- packages/matplotlib/axes.py in update_datalim(self, xys) 966 # and the data in xydata 967 xys = asarray(xys) --> 968 self.dataLim.update_numerix_xy(xys, -1) 969 970 : Bbox::update_numerix_xy expected numerix array WARNING: Failure executing file: Rob Hetland, Associate Professor Dept. of Oceanography, Texas A&M University http://pong.tamu.edu/~rob phone: 979-458-0096, fax: 979-845-6331 - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] *EGG* now freezes at this same place always....
[EMAIL PROTECTED] wrote: > Thanks for help. Now it freezes always here... > > > GTK requires pygtk > GTKAgg requires pygtk > TKAgg requires TkInter I never used eggs, but I guess you need to install these libs by yourself. apt-cache search for this stuff and make sure you install the *-dev versions of the packages. Seems that you need (list may not exhaustive) python-gtk2-dev (pygtk) and tk8.4-dev, python-tk, ... (TkInter, this is a little tricky to search for, check the dependencies) -- cheers, steve Random number generation is the art of producing pure gibberish as quickly as possible. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] *EGG* now freezes at this same place always....
Thanks for help. Now it freezes always here... GTK requires pygtk GTKAgg requires pygtk TKAgg requires TkInter warning: no files found matching 'MANIFEST' warning: no files found matching 'lib/matplotlib/toolkits' no previously-included directories found matching 'examples/_tmp_*' In file included from /usr/include/python2.4/Python.h:8, from CXX/Objects.hxx:9, from CXX/Extensions.hxx:19, from src/_transforms.h:12, from src/_ns_transforms.cpp:5: /usr/include/python2.4/pyconfig.h:832:1: warning: "_POSIX_C_SOURCE" redefined In file included from /usr/include/c++/3.3/i486-linux/bits/os_defines.h:39, from /usr/include/c++/3.3/i486-linux/bits/c++config.h:35, from /usr/include/c++/3.3/functional:53, from src/_ns_transforms.cpp:1: /usr/include/features.h:131:1: warning: this is the location of the previous definition Chris - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
Eric Firing wrote: > Regarding the clip line, I think that your test for mask is None is not > the right solution because it knocks out the clipping operation, but the > clipping is intended regardless of the state of the mask. I had > expected it to be a very fast operation, for what it's worth, a few years ago a wrote a "fast_clip" c extension that did clip without making nearly as many temporary arrays as the Numeric one -- I don't know what numpy does , I haven't needed a fast clip recently. I'd be glad to send the code to anyone interested. > Now I recall very recent discussion explaining why "where" is slow > compared to indexing with a boolean, so I know I can speed it up with > numpy. Unfortunately Numeric does not support this, so maybe what will > be needed is numerix functions that take advantage of numpy when > available. good idea. > This is one of those times when I really wish we could drop > Numeric and numarray support *now* and start taking full advantage of numpy. I'd love that too. Maybe your proposal is a good one, though -- make numeric functions that are optimized for numpy. I think that's a good way to transition. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [EMAIL PROTECTED] - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] broken images on tutorial
When I look at http://matplotlib.sourceforge.net/tutorial.html with Safari, I see a lot of broken images. Any ideas? smime.p7s Description: S/MIME cryptographic signature - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
David, >- first, we can see that in expose_event (one is expensive, the other > negligeable, from my understanding), two calls are pretty expensive: > the __call__ at line 735 (for normalize functor) and one for __call__ > at line 568 (for colormap functor). >- for normalize functor, one line is expensive: val = > ma.array(clip(val.filled(vmax), vmin, vmax), mask=mask). If I put a test > on mask when mask is None (which it is in my case), then the function > becomes negligeable. >- for colormap functor, the 3 where calls are expensive. I am not > sure to understand in which case they are useful; if I understand > correctly, one tries to avoid > values out of range (0, N), and force out of range values to be clipped. > Isn't there an easier way than using where ? > >If I remove the where in the colormap functor, I have a 4x speed > increase for the to_rgba function. After that, it becomes a bit more > tricky to change things for someone like me who have no knowledge about > matplotlib internals. The things you have identified were added by me to support masked array bad values and special colors for regions above or below the mapped range of values. I will be happy to make changes to speed them up. Regarding the clip line, I think that your test for mask is None is not the right solution because it knocks out the clipping operation, but the clipping is intended regardless of the state of the mask. I had expected it to be a very fast operation, so I am surprised it is a bottleneck; in any case I can take a look to see how it can be sped up, or whether it can be bypassed in some cases. Maybe it is also using "where" internally. Now I recall very recent discussion explaining why "where" is slow compared to indexing with a boolean, so I know I can speed it up with numpy. Unfortunately Numeric does not support this, so maybe what will be needed is numerix functions that take advantage of numpy when available. This is one of those times when I really wish we could drop Numeric and numarray support *now* and start taking full advantage of numpy. In any case, thanks for pointing out the slowdowns--I will fix them as best I can--and keep at it. I share your interest in speeding up interactive use of matplotlib, along with fixing bugs, filling holes in functionalisy, and smoothing rough edges. There is a lot to be done. As John noted, though, there will always be tradeoffs among flexibility, code simplicity, generality, and speed. Eric - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] font_manager.py gives following traceback
I have received reports from clients with the following traceback or similar. This happens when application is packaged with py2exe. Traceback (most recent call last): File "appwine.pyo", line 1341, in OnToolbarChart File "frameplotmpl.pyo", line 19, in ? File "matplotlib\backends\__init__.pyo", line 19, in ? File "matplotlib\backends\backend_wxagg.pyo", line 18, in ? File "matplotlib\backends\backend_agg.pyo", line 82, in ? File "matplotlib\figure.pyo", line 3, in ? File "matplotlib\axes.pyo", line 14, in ? File "matplotlib\axis.pyo", line 21, in ? File "matplotlib\font_manager.pyo", line 982, in ? File "matplotlib\font_manager.pyo", line 826, in __init__ File "matplotlib\font_manager.pyo", line 819, in rebuild File "matplotlib\font_manager.pyo", line 454, in createFontDict SystemError: error return without exception set Any hints of what might cause this would be very welcome. Werner P.S. I am still on matplotlib version '0.82' (plan to upgrade to newer version but need to upgrade to Unicode wxPython first), with Python 2.4 and wxPython 2.6.x - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] matplotlib *EGG* installation works on Fedora but not Debian Etch..
I don't think this has anything to do with eggs. It looks like you don't have a C++ compiler installed or configured correctly. On ubuntu/debian you should make sure "build-essentials" is installed. On 12/13/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Yes we all know the normal install of Matplotlib is rock solid and reliable. > > I'm having trouble doing an "egg" (setuptools) install of matplotlib. > > (I'm hoping eggs will be a nice way to have uniform install instructions > across > all OSes.) > > I got numpy egg installed but got this when I tried matplotlib egg install > > gcc: installation problem, cannot exec `cc1plus': No such file or directory > gcc: installation problem, cannot exec `cc1plus': No such file or directory > error: Setup script exited with error: Command "gcc -pthread > -fno-strict-aliasin g -DNDEBUG > -g > -O3 -Wall -Wstrict-prototypes -fPIC -Iagg23/include -Isrc -Iswig - > I/usr/include/python2.4 -c agg23/src/agg_trans_affine.cpp -o > build/temp.linux-i6 > 86-2.4/agg23/src/agg_trans_affine.o" failed with exit status 1 > Exception exceptions.OSError: (2, 'No such file or directory', > 'src/_ns_cntr.c')in method CleanUpFile.__del__ of d59ac>> ignored > Exception exceptions.OSError: (2, 'No such file or directory', > 'src/_ns_backend_ agg.cpp') in > at 0xb78d53cc>> ignored > Exception exceptions.OSError: (2, 'No such file or directory', > 'src/_ns_nxutils. c') in method CleanUpFile.__del__ of b78d5b8c>> ignored > Exception exceptions.OSError: (2, 'No such file or directory', > 'src/_ns_image.cp p') in method CleanUpFile.__del__ of b78d57ac>> ignored > Exception exceptions.OSError: (2, 'No such file or directory', > 'src/_ns_transfor ms.cpp') in > at 0xb796c12c>> ignored > > > Any help greatly appreciated. > > Chris > > - > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > ___ > Matplotlib-users mailing list > Matplotlib-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Some remarks/questions about perceived slowness of matplotlib
David Cournapeau wrote: > But the show case is more interesting: > > ncalls tottime percall cumtime percall filename:lineno(function) > 10.0020.0023.8863.886 > slowmatplotlib.py:177(bench_imshow_show) > 10.0000.0003.8843.884 > slowmatplotlib.py:163(bench_imshow) > 10.6980.6983.0033.003 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py:70(show) > > > 20.0000.0002.2661.133 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py:275(expose_event) > > > 10.0090.0092.2662.266 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_gtkagg.py:71(_render_figure) > > > 10.0000.0002.2562.256 > /home/david/local/lib/python2.4/site-packages/matplotlib/backends/backend_agg.py:385(draw) > > > 10.0000.0002.2532.253 > /home/david/local/lib/python2.4/site-packages/matplotlib/figure.py:510(draw) > > 10.0000.0002.2512.251 > /home/david/local/lib/python2.4/site-packages/matplotlib/axes.py:994(draw) > > 10.0050.0051.9511.951 > /home/david/local/lib/python2.4/site-packages/matplotlib/image.py:173(draw) > > 10.0960.0961.9461.946 > /home/david/local/lib/python2.4/site-packages/matplotlib/image.py:109(make_image) > > > 10.0020.0021.8501.850 > /home/david/local/lib/python2.4/site-packages/matplotlib/cm.py:50(to_rgba) > > 10.0010.0010.9490.949 > /home/david/local/lib/python2.4/site-packages/matplotlib/colors.py:735(__call__) > > > 10.0970.0970.8990.899 > /home/david/local/lib/python2.4/site-packages/matplotlib/colors.py:568(__call__) > > > 3250.0500.0000.6710.002 > /home/david/local/lib/python2.4/site-packages/numpy/core/ma.py:533(__init__) > > 10.6000.6000.6000.600 > /home/david/local/lib/python2.4/site-packages/numpy/core/fromnumeric.py:282(resize) > > > 10.0000.0000.5960.596 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:1894(imshow) > > > 100.5700.0570.5700.057 > /home/david/local/lib/python2.4/site-packages/numpy/oldnumeric/functions.py:117(where) > > > 30.0000.0000.5130.171 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:883(gca) > > 10.0000.0000.5130.513 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:950(ishold) > > 40.0000.0000.4080.102 > /home/david/local/lib/python2.4/site-packages/matplotlib/pylab.py:903(gcf) > > > For more details, see the .kc files which are the in the tbz2 archive, > with the script for generating profiles for kcachegrind, Here is some stuff I tried: - first, we can see that in expose_event (one is expensive, the other negligeable, from my understanding), two calls are pretty expensive: the __call__ at line 735 (for normalize functor) and one for __call__ at line 568 (for colormap functor). - for normalize functor, one line is expensive: val = ma.array(clip(val.filled(vmax), vmin, vmax), mask=mask). If I put a test on mask when mask is None (which it is in my case), then the function becomes negligeable. - for colormap functor, the 3 where calls are expensive. I am not sure to understand in which case they are useful; if I understand correctly, one tries to avoid values out of range (0, N), and force out of range values to be clipped. Isn't there an easier way than using where ? If I remove the where in the colormap functor, I have a 4x speed increase for the to_rgba function. After that, it becomes a bit more tricky to change things for someone like me who have no knowledge about matplotlib internals. Cheers, David - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
