Re: [Gimp-developer] Suggested in memory image data structure.
On Mon, Jul 07, 2008 at 08:45:04AM +0200, Sven Neumann wrote: > But as soon as you perform an operation on the image, the > full-scale image data needs to be touched. Agreed. This is much easier to program. However, when I'm zoomed in to say a 4 megapixel part of the image, and the selection is in view, then if I do something a max of 4 Mpixels, of 4 bytes * 2 need to be touched. I have plenty of RAM for that. > It's non-trivial to avoid this. But it is on the long-term plan to > implement working on the scaled-down version. We hope to be able to > address this when GIMP makes full use of GEGL. If you want to help, > I suggest that you contribute to the GEGL development. So, when on my original image, I select part, and use the "fill with gradient" tool, I first get a progress bar that moves in about 30 seconds from 0 to 100%. Then I have to wait about 30 seconds for the swapping to stop. The fill could be done on the small scale cache, allowing the GUI to be updated and then a background process can update the actual image, without me having to wait for it to complete. Is that similar to what GEGL intends to do? Roger. -- ** [EMAIL PROTECTED] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** **Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. - Adapted from lxrbot FAQ ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Sun, Jul 06, 2008 at 09:54:50PM -0700, Bill Skaggs wrote: > On Sun, Jul 6, 2008 at 9:41 PM, Rogier Wolff <[EMAIL PROTECTED]> wrote: > > > But then why, on a machine with 700Mb RAM, does EVERYTHING take > > so very long? My image is "only 50Mpixel". > The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and > that's if you only have a single layer. So given the amount of > memory needed for other purposes in a modern operating system, it's > likely that gimp is swapping very frequently. 700 MB is low to be > running Gimp in the first place, and much more so if you're going to > work with large images. Gimp is going in directions that should > eventually make it work better in this kind of scenario, but for now > and the near future, the best advice is to get more RAM. -- Bill OK. I've scaled my image down by a factor of three on both axes, so I now have a workable amount of pixels/RAM. I forgot to photograph part of the sky. So I need to stitch some blue to the image I have. I've used the color picker, and the fill gradient tool to create some sky pixels that look almost ok. If I'm zoomed in a bit, and somehow the whole image is selected, if I try to create a selection, I always "move the selection" instead of creating a new selection. I have to zoom out to be able to "invite" part of "outside the image" into view, so that I can click outside the image to start a new selection. There is probably something I'm missing here. Next, the line between the photographed sky and the gradient is sometimes visible. So I want to smudge this a bit. First I thought there was a tool that would even out the pixels under the cursor, to do this. However the smudge tool seems to work differently. I have to drag to create a source and then let go where I want to drag those pixels. Fine. So click on one side of the "edge", drag to the edge, let go. (with the biggest fuzzy cicle tool). After doing that two or three times, my view suddenly shifts, the middle of the image is in view, and apparently I've smudged a whole horizontal line. Part of my picture is completely smudged over the whole width of the image. It happens every three or four times I try to smudge that edge away. Is this a bug? Or am I missing something again? Roger. -- ** [EMAIL PROTECTED] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** **Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. - Adapted from lxrbot FAQ ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
Hi, On Mon, 2008-07-07 at 08:00 +0200, Rogier Wolff wrote: > > The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and > > Fine. So you're saying that gimp wastes a factor of two of > memory. Fine! probably neccesary for something. No problem with that. GIMP needs to allocate the memory to hold your image data and it needs to keep the projection (that's what you see with all layers merged) in memory. On top of that there's some extra memory for the selection mask. We could save the memory for the projection in the special case of a single-layered image with the layer mode set to Normal and 100% opacity. There's even code to share the tiles in this special case but it had to be disabled because it is broken. Perhaps someone can fix it one day, that would be a nice improvement. > So, I understand that loading the big image takes some time and > swapping. Then, moving the viewable window to the area I want to > change may cause some swapping. But from then on, things should be > working snappy. However everything I do takes lots of time. All operations are applied on the full image data. We use a mipmap for the projection which helps a lot for panning through zoomed-out views of large images. But as soon as you perform an operation on the image, the full-scale image data needs to be touched. It's non-trivial to avoid this. But it is on the long-term plan to implement working on the scaled-down version. We hope to be able to address this when GIMP makes full use of GEGL. If you want to help, I suggest that you contribute to the GEGL development. Sven ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Sun, Jul 06, 2008 at 09:54:50PM -0700, Bill Skaggs wrote: > On Sun, Jul 6, 2008 at 9:41 PM, Rogier Wolff <[EMAIL PROTECTED]> wrote: > > > But then why, on a machine with 700Mb RAM, does EVERYTHING take > > so very long? My image is "only 50Mpixel". > The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and Fine. So you're saying that gimp wastes a factor of two of memory. Fine! probably neccesary for something. No problem with that. > that's if you only have a single layer. So given the amount of > memory needed for other purposes in a modern operating system, it's > likely that gimp is swapping very frequently. 700 MB is low to be > running Gimp in the first place, and much more so if you're going to > work with large images. Gimp is going in directions that should > eventually make it work better in this kind of scenario, but for now > and the near future, the best advice is to get more RAM. So What I started out saying is that my brain is not prepared to look at those 50 megapixels at one time. My screen is only 1.3 megapixels. So, I understand that loading the big image takes some time and swapping. Then, moving the viewable window to the area I want to change may cause some swapping. But from then on, things should be working snappy. However everything I do takes lots of time. Indeed for the short run, I could buy more memory. But this computer does not earn me money doing GIMP. It's fine for all the other stuff that I bought it for. If I want I can probably use the machine with 8G at work for this. What apparently happens in the GIMP developper community is that everyone buys loads of extra memory, and then the developpers don't feel the pain of GIMP's inefficient use of memory. So, is the tiling very inefficient? Do the parameters differ enormously from what I suggested? Does gimp reference the whole image with every thing I do? If so, that would explain why I seem to need the whole image in memory, while I'm only working on a small part. Roger. -- ** [EMAIL PROTECTED] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** **Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. - Adapted from lxrbot FAQ ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Sun, Jul 6, 2008 at 9:41 PM, Rogier Wolff <[EMAIL PROTECTED]> wrote: > But then why, on a machine with 700Mb RAM, does EVERYTHING take > so very long? My image is "only 50Mpixel". The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and that's if you only have a single layer. So given the amount of memory needed for other purposes in a modern operating system, it's likely that gimp is swapping very frequently. 700 MB is low to be running Gimp in the first place, and much more so if you're going to work with large images. Gimp is going in directions that should eventually make it work better in this kind of scenario, but for now and the near future, the best advice is to get more RAM. -- Bill ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Mon, 2008-07-07 at 06:41 +0200, Rogier Wolff wrote: > On Sun, Jul 06, 2008 at 05:51:24PM -0400, Liam R E Quin wrote: > > > Currently images are commonly stored as an array of rows of pixels. > > GIMP does in fact use image tiles. > > But then why, on a machine with 700Mb RAM, does EVERYTHING take > so very long? My image is "only 50Mpixel". > > Roger. What size is your tile cache? Stephen ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Sun, Jul 06, 2008 at 05:51:24PM -0400, Liam R E Quin wrote: > > Currently images are commonly stored as an array of rows of pixels. > GIMP does in fact use image tiles. But then why, on a machine with 700Mb RAM, does EVERYTHING take so very long? My image is "only 50Mpixel". Roger. -- ** [EMAIL PROTECTED] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** **Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. - Adapted from lxrbot FAQ ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
Rogier Wolff wrote: > Hi, > > Currently images are commonly stored as an array of rows of pixels. > > I just realized I missed the word 'commonly', so I would like to adjust my answer to: Yes but not in any serious image editing application like GIMP. In GIMP images and layers etc are stored in tile based structures. - Martin ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
Rogier Wolff wrote: > Hi, > > Currently images are commonly stored as an array of rows of pixels. > Hi No, they are stored in a tile based structure, check out app/base/tile-manager.h > [Description of mipmapping and a wish to implement it in GIMP] > The projection is mipmapped since GIMP 2.4, check out app/base/tile-pyramid.h Best regards, Martin Nordholts ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Sun, 2008-07-06 at 23:34 +0200, Rogier Wolff wrote: > I'm trying to touch up the result of a panorama shot I took. This > takes quite long, mostly because the image is too big to fit into > memory. Working with large images does require a little extra care. It helps to set the gimp tile cache (in preferences) to a fairly large value. E.g. I use a gigabyte or more. But when I bought extra RAM for my desktop, it made a huge difference. > Currently images are commonly stored as an array of rows of pixels. GIMP does in fact use image tiles. Best, Liam -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org www.advogato.org ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Suggested in memory image data structure.
On Sun, 6 Jul 2008 23:34:40 +0200 Rogier wrote: > I'm trying to touch up the result of a panorama shot I took. This > takes quite long, mostly because the image is too big to fit into > memory. nice one Roger, I did have great problems on the same issue with sat maps with an older version of gimp (2.2 if I remember correctly), since then I gave up. I never inspected that deep as you did, so I don't even know if the svn version has any improvements (but I noticed a speedup in applying an image manipulation filter). -- Matteo 'Peach' Pescarin ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
[Gimp-developer] Suggested in memory image data structure.
Hi, I'm trying to touch up the result of a panorama shot I took. This takes quite long, mostly because the image is too big to fit into memory. I would like to suggest a way of storing the image data in memory in such a way that it doesn't take so much memory. Or at least, it doesn't require so much memory to be presently paged in. Currently images are commonly stored as an array of rows of pixels. With an image width of more than 1400 pixels, if I need to access a vertical line of pixels, this will touch a different machine memory page for every single line. Thus to access say 100 pixels, 400 kbytes of core memory needs to be touched (from swap if memory is tight). First, a page on a machine is usually about 4k. So the basic units might be 4k. On the other hand, having the units be say 12k doesn't cost much in terms of IO time, but makes things in binary fit a lot easier. So, instead of storing images in an array of rows of pixels, we can store an image as an array of blocks. If a block is 64x64 pixels, with 24bit data, the block takes 12k of memory. Now, touching 100 vertical pixels requires only two blocks of 12k to be in memory. This is a big improvement. Next, if we want to display an image that is say 5000x12000, my screen has only one or two megapixels. So scanning all 60 megapixels to display this is a waste of CPU time and memory. To improve this, several scaled down versions (say by a factor of two each time) can be kept. Thus of my 5000x12000 image, a 2500x6000, 1250x3000, and a 625x1500 version should be kept. To display up to 625 vertical pixels of my full image, should use the last version. This means that less than 3 megabytes of memory needs to be touched to display my image. On a reasonable computer this is unnoticably fast. As each scaled down version uses 4x less memory, they cost little to build and to maintain. Oh. If I zoom into a 100x100 area of my big (5000x12000) image, the old way of storing images would require 100 or 200 pages of image memory to be present in core. My suggestion only 2 or 4. If all plugins and internal image manipulation functions are "aware" of the old way of storing images, then I'm afraid that this is a major programming undertaking. If however, everybody accesses the image through accessfunctions (getpixel, putpixel?) then things are a whole lot easier. Once you start manipulating images, of course, my suggestion makes things a bit slower. However, I seem to be waiting a lot for my drive instead of "doing what humans do best", so I think this would mean a speedup on the whole. But of course, if things easily fit into memory it might slow things down a bit. So possibly a runtime switch between the storage methods is desirable. Roger. -- ** [EMAIL PROTECTED] ** http://www.BitWizard.nl/ ** +31-15-2600998 ** **Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233** *-- BitWizard writes Linux device drivers for any device you may have! --* Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. - Adapted from lxrbot FAQ ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Menus under Help/User Manual
Hi, On Sun, 2008-07-06 at 10:41 +0200, Roman Joost wrote: > I was about going through the pointers we added to the interesting > topics of the manual. Those links like "Basic Concepts" were added to > the menu to help users reading about things they might need to start > when using GIMP. Long term we would rather like to get rid of this menu entirely. IMO it would be much better if we included links to the user manual in the GIMP tips. We have some plans for GIMP 2.8 to improve the tips system and to put it into a more prominent location (the empty image window). > The manual has a few more interesting topics where I'd like to add links > to. So I've two questions: > > Since it's a script-fu script, I'd like to know if that is going to > be migrated to an XML menu file? It's a script because that was the simplest solution. Making it an XML file would require writing a plug-in that parses this file and installs the menu items. > Unfortunately the menu is ordered alphabetically, which for the > translated version looks kind of messed up. At first putting numbers > in front on the menu entries came to my mind, which seems like a > bad workaround on another thought. Is it possible to have some kind > of ordering. Though probably problematic if it should stay the same > with all the translations of the menu. Plug-in menu entries are sorted alphabetically and plug-ins can't do anything about that ordering. Why does this look messed up for translated versions? Sven ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
[Gimp-developer] Menus under Help/User Manual
Hi, I was about going through the pointers we added to the interesting topics of the manual. Those links like "Basic Concepts" were added to the menu to help users reading about things they might need to start when using GIMP. The manual has a few more interesting topics where I'd like to add links to. So I've two questions: Since it's a script-fu script, I'd like to know if that is going to be migrated to an XML menu file? Unfortunately the menu is ordered alphabetically, which for the translated version looks kind of messed up. At first putting numbers in front on the menu entries came to my mind, which seems like a bad workaround on another thought. Is it possible to have some kind of ordering. Though probably problematic if it should stay the same with all the translations of the menu. Cheers, -- Roman Joost www: http://www.romanofski.de email: [EMAIL PROTECTED] signature.asc Description: Digital signature ___ Gimp-developer mailing list Gimp-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gimp-developer
