[Matplotlib-users] plotting large images
Hi, I would like to plot multiple overlayed 4096x4096 images in one axes. If I run this code the plot takes 300 MB of memory: import numpy as np import matplotlib.pyplot as plt if __name__ == '__main__': img = np.zeros((4096, 4096)) img[100: 300, 100:1500] = 200 imgplot = plt.imshow(img) plt.show() And it takes additional 300 MB for every image with this size added into plot. Is there any way to reduce memory consumption without need of data resampling? My configuration: Matplotlib 1.2.1 Numpy 1.7.1 Ubuntu 13.04 64 bit Best Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting large images
You could, before plotting, sum the different image arrays? Depending on whether you are plotting RGB(A) images or greyscale images, you could take the sum of the color channels, or take a weighted average. The method you use here depends strongly on the image type, but it will reduce memory consumption. Just a thought. 2013/8/27 Štěpán Turek stepan.tu...@seznam.cz Hi, I would like to plot multiple overlayed 4096x4096 images in one axes. If I run this code the plot takes 300 MB of memory: import numpy as np import matplotlib.pyplot as plt if __name__ == '__main__': img = np.zeros((4096, 4096)) img[100: 300, 100:1500] = 200 imgplot = plt.imshow(img) plt.show() And it takes additional 300 MB for every image with this size added into plot. Is there any way to reduce memory consumption without need of data resampling? My configuration: Matplotlib 1.2.1 Numpy 1.7.1 Ubuntu 13.04 64 bit Best Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Graph ticks label missing !
Le Lun 26 août 2013 18:21, Goyo a écrit : 2013/7/19 Nicolas Mailhot nicolas.mail...@laposte.net: Le Mer 17 juillet 2013 14:56, Michael Droettboom a écrit : Can you please provide a completely standalone example? The following code has undefined variables etc. Here it is, I'm afraid this testcase intent is less clear than what I pasted previously (I replaced variables with precomputed values) As shown in the attached png, the bottom tick labels (month names) are missing. It worked in matplotlib ≤ 1.2.0 I can confirm the issue with 1.2.1 but it works with a recent development version (output attached) so it must have been fixed at some point. Thank you very much for the data point, I'll try to get 1.3.0 built on RHEL 6 Regards, -- Nicolas Mailhot -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting large images
Hi, You could, before plotting, sum the different image arrays? Depending on whether you are plotting RGB(A) images or greyscale images, you could take the sum of the color channels, or take a weighted average. Yes, I will probably merge the images (RGBA) before plotting. I want to create more plots and even with this optimization every plot will take 300 MB... Is there any way how to save some memory? Best Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting large images
Those numbers actually make a lot of sense. For a 4k by 4k 2D array of 64-bit floats, you're using 128MiB of memory, just to store them. Displaying such an array with mpl would take a copy of that and add some objects for housekeeping (on my machine about 150MB to display one such array together with the housekeeping objects). You could look at whether or not you actually need 64-bit precision. Often times, 8-bit precision per color channel is justifiable, even in grayscale. My advice is to play with the dtype of your array or, as you mentioned, resample. Also, is it needed to keep all images? It sounds to me like your application will become very resource hungry if you're going to be displaying several of these 2D images over each other (and if you don't use transparency, you won't get any benefit at all from plotting them together). 2013/8/27 Štěpán Turek stepan.tu...@seznam.cz Hi, You could, before plotting, sum the different image arrays? Depending on whether you are plotting RGB(A) images or greyscale images, you could take the sum of the color channels, or take a weighted average. Yes, I will probably merge the images (RGBA) before plotting. I want to create more plots and even with this optimization every plot will take 300 MB... Is there any way how to save some memory? Best Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting large images
You could look at whether or not you actually need 64-bit precision. Often times, 8-bit precision per color channel is justifiable, even in grayscale. My advice is to play with the dtype of your array or, as you mentioned, resample. thanks, this helped me significantly, uint8 precision is enough. Also, is it needed to keep all images? It sounds to me like your application will become very resource hungry if you're going to be displaying several of these 2D images over each other (and if you don't use transparency, you won' t get any benefit at all from plotting them together). Yes, I need them all . To avoid it I am thinking about merging them into one image and then plot it. Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting large images
I've been burned by this before as well. MPL stores some intermediate data products (for example, scaled RGB copies) at full resolution, even though the final rendered image is downsampled depending on screen resolution. I've used some hacky tricks to get around this, which mostly involve downsampling the image on the fly based on screen resolution. One such effort is at https://github.com/ChrisBeaumont/mpl-modest-image. If you are loading your arrays from disk, you can also use memory-mapped arrays -- this prevents you from loading all the data into RAM, and further cuts down on the footprint. cheers, chris On Tue, Aug 27, 2013 at 6:49 AM, Štěpán Turek stepan.tu...@seznam.czwrote: You could look at whether or not you actually need 64-bit precision. Often times, 8-bit precision per color channel is justifiable, even in grayscale. My advice is to play with the dtype of your array or, as you mentioned, resample. thanks, this helped me significantly, uint8 precision is enough. Also, is it needed to keep all images? It sounds to me like your application will become very resource hungry if you're going to be displaying several of these 2D images over each other (and if you don't use transparency, you won't get any benefit at all from plotting them together). Yes, I need them all . To avoid it I am thinking about merging them into one image and then plot it. Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting large images
On 08/27/2013 09:49 AM, Chris Beaumont wrote: I've been burned by this before as well. MPL stores some intermediate data products (for example, scaled RGB copies) at full resolution, even though the final rendered image is downsampled depending on screen resolution. I've used some hacky tricks to get around this, which mostly involve downsampling the image on the fly based on screen resolution. One such effort is at https://github.com/ChrisBeaumont/mpl-modest-image. It looks like this wouldn't be too hard to include in matplotlib. I don't think we'd want to change the current behavior, because sometimes its tradeoff curve makes sense, but in other cases, the modest image approach also makes sense. It's just a matter of coming up with an API to switch between the two behaviors. Pull request? Cheers, Mike If you are loading your arrays from disk, you can also use memory-mapped arrays -- this prevents you from loading all the data into RAM, and further cuts down on the footprint. cheers, chris On Tue, Aug 27, 2013 at 6:49 AM, S(te(pán Turek stepan.tu...@seznam.cz mailto:stepan.tu...@seznam.cz wrote: You could look at whether or not you actually need 64-bit precision. Often times, 8-bit precision per color channel is justifiable, even in grayscale. My advice is to play with the dtype of your array or, as you mentioned, resample. thanks, this helped me significantly, uint8 precision is enough. Also, is it needed to keep all images? It sounds to me like your application will become very resource hungry if you're going to be displaying several of these 2D images over each other (and if you don't use transparency, you won't get any benefit at all from plotting them together). Yes, I need them all . To avoid it I am thinking about merging them into one image and then plot it. Stepan -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net mailto:Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58040911iu=/4140/ostg.clktrk___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
