Re: An alternative to gdk-pixbuf
On 09/15/18 10:48 AM, John Emmas wrote: On 15/09/2018 12:07, Magnus Bergman wrote: Some mismatch in versions of libtiff could for example be a reason. Otherwise you should probably file a bug report with more information (perhaps to your distribution firstly). Do you happen to know if the tiff library has its own mailing list? I haven't had much success in finding one Yes - see http://www.simplesystems.org/libtiff/ for info on the list. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - https://blogs.oracle.com/alanc ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 16/09/2018 11:18, Emmanuele Bassi wrote: The correct way to report issues for gdk-pixbuf: 1. use the issue tracker: https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/new Thanks Emmanuele - done! ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Sun, 16 Sep 2018 at 10:47, John Emmas wrote: > On 15/09/2018 18:48, John Emmas wrote: > > > > Do you happen to know if the tiff library has its own mailing list? I > > haven't had much success in finding one > > > > In fact I'll need the mailing list for gdk-pixbuf now - except that I > can't find that one either!! (or is this the correct place for reporting > gdk-pixbuf issues?) > > The correct way to report issues for gdk-pixbuf: 1. use the issue tracker: https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/new 2. do not attach to an existing, unrelated thread on a mailing list; create a new topic instead For discussions, you can use this mailing list—but issues should strictly be reported on the issue tracker. Ciao, Emmanuele. -- https://www.bassi.io [@] ebassi [@gmail.com] ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 15/09/2018 18:48, John Emmas wrote: Thanks Magnus. I've a feeling that the problem might come down to struct alignment. No, I was wrong about that. I've tracked the problem to commit #ce52cefbbc in gdk-pixbuf (which brings me to the 2nd problem...) On 15/09/2018 18:48, John Emmas wrote: Do you happen to know if the tiff library has its own mailing list? I haven't had much success in finding one In fact I'll need the mailing list for gdk-pixbuf now - except that I can't find that one either!! (or is this the correct place for reporting gdk-pixbuf issues?) John ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 15/09/2018 12:07, Magnus Bergman wrote: Some mismatch in versions of libtiff could for example be a reason. Otherwise you should probably file a bug report with more information (perhaps to your distribution firstly). Thanks Magnus. I've a feeling that the problem might come down to struct alignment. Although I'm building on Windows, I currently compile both tiff and gdk-pixbuf with 1-byte struct alignment which seems to work for png / jpeg etc. But maybe tiff expects 8-byte alignment? Do you happen to know if the tiff library has its own mailing list? I haven't had much success in finding one John ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Sat, 15 Sep 2018 09:47:33 +0100 John Emmas wrote: > Sorry, I haven't been following this conversation but as a > side-issue... I only noticed this morning that gdk-pixbuf doesn't > seem to be able to load TIF images any more. I've attached a small > file that won't load but I haven't managed to load any TIF image from > the ones I've tested this morning. > > If someone's got a few spare minutes, could they try loading the > attached image using 'gdk_pixbuf_new_from_file()' and just let me > know if it works? I'm sure this always worked for me in the > past... :-( The loader hasn't been removed from gdk-pixbuf so it should work. What you can do to trouble shoot is to run gdk-pixbuf-query-loaders and see if the TIFF loader is included. You may notice some information about the it and why it can't load. Some mismatch in versions of libtiff could for example be a reason. Otherwise you should probably file a bug report with more information (perhaps to your distribution firstly). ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Sorry, I haven't been following this conversation but as a side-issue... I only noticed this morning that gdk-pixbuf doesn't seem to be able to load TIF images any more. I've attached a small file that won't load but I haven't managed to load any TIF image from the ones I've tested this morning. If someone's got a few spare minutes, could they try loading the attached image using 'gdk_pixbuf_new_from_file()' and just let me know if it works? I'm sure this always worked for me in the past... :-( John ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Tue, 11 Sep 2018 13:22:17 +0200 Bastien Nocera wrote: > On Tue, 2018-09-11 at 07:40 +0100, John Cupitt via gtk-devel-list > wrote: > > On Tue, 11 Sep 2018 at 03:11, Magnus Bergman > > wrote: > > > On Tue, 11 Sep 2018 00:07:27 +0200 > > > Bastien Nocera wrote: > > > > No, it really isn't: > > > > > https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html > > > > > > > > We want to have less CVEs, not more. > > > > > > I see what you mean. A few of them (although none of the more > > > serious > > > ones) were even related to the GIF loader specifically. But the > > > sheer > > > volume kind of speaks for itself otherwise. :( > > > > IM joined Google's OSS-Fuzz programme last year: > > > > https://github.com/google/oss-fuzz > > > > The huge surge in CVEs was caused by that --- they've been fixing > > one or two a day ever since. Once they are through this very painful > > process, IM ought to be rather safe. > > > > I do agree though that it's a large and complex thing to use for > > such a (relatively) simple task. > > I maintained ImageMagick in RHEL a long time ago, it was already that > way though security issues cropped up a bit less often than every day > (!). I don't see any reason for us to want to us it. Understandable. Anyway, I made a third GIF loader based on libnsgif and released it. This one also handles all images in the test suit and displays 1_partyanimsm2.gif nicely too (before I even found something that could view it I wasn't sure if it was supposed to show garbage or not). Since libnsgif is very straight forward to use I guess you prefer to write your own loader instead of having abydos in the middle. If you don't already know the API of libnsgif (it lacks documentation) it is probably very easy to get started by porting the abydos plugin to gdk-pixbuf. And here are some tips about things you might want to implement differently: The pixel format libnsgif uses happens to be the same as for gdk-pixbuf (both for big and little endian). This means that the bitmap_create callback can be used to create a GdkPixbuf and bítmap_get_buffer to return the pixels of the GdkPixbuf, so libnsgif can draw directly into it. However this assumes that the rowstride equals the width (which I guess is a valid assumption for RGBA). It would be easy just returning a copy of this GdkPixbuf then requested and use gif_decode_frame() to change the frame then needed (obviously using locking since several independent GdkPixbufAnimationIter could theoretically be used at the same time even if it's a highly unlikely use case). This will possibly make it even simpler and more readable than the approach I took with the abydos loader. The reason I chose to decode all frames directly upon load and then throw away the gif_animation was manly because the conversion of abydos doesn't require the pixels to be copied. Also note that progressive loading is supported by libnsgif, but instead of just feeding it the new bytes, it needs a buffer containing all the previous bytes as well. And you probably want to add support for (netscape style) looping, which is something abydos doesn't support yet. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Tue, 2018-09-11 at 07:40 +0100, John Cupitt via gtk-devel-list wrote: > On Tue, 11 Sep 2018 at 03:11, Magnus Bergman > wrote: > > On Tue, 11 Sep 2018 00:07:27 +0200 > > Bastien Nocera wrote: > > > No, it really isn't: > > > https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html > > > > > > We want to have less CVEs, not more. > > > > I see what you mean. A few of them (although none of the more > > serious > > ones) were even related to the GIF loader specifically. But the > > sheer > > volume kind of speaks for itself otherwise. :( > > IM joined Google's OSS-Fuzz programme last year: > > https://github.com/google/oss-fuzz > > The huge surge in CVEs was caused by that --- they've been fixing one > or two a day ever since. Once they are through this very painful > process, IM ought to be rather safe. > > I do agree though that it's a large and complex thing to use for such > a (relatively) simple task. I maintained ImageMagick in RHEL a long time ago, it was already that way though security issues cropped up a bit less often than every day (!). I don't see any reason for us to want to us it. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Tue, 11 Sep 2018 at 03:11, Magnus Bergman wrote: > On Tue, 11 Sep 2018 00:07:27 +0200 > Bastien Nocera wrote: > > No, it really isn't: > > https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html > > > > We want to have less CVEs, not more. > > I see what you mean. A few of them (although none of the more serious > ones) were even related to the GIF loader specifically. But the sheer > volume kind of speaks for itself otherwise. :( IM joined Google's OSS-Fuzz programme last year: https://github.com/google/oss-fuzz The huge surge in CVEs was caused by that --- they've been fixing one or two a day ever since. Once they are through this very painful process, IM ought to be rather safe. I do agree though that it's a large and complex thing to use for such a (relatively) simple task. John ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Tue, 11 Sep 2018 00:07:27 +0200 Bastien Nocera wrote: > On Mon, 2018-09-10 at 22:29 +0200, Magnus Bergman wrote: > > On Mon, 10 Sep 2018 11:31:42 +0200 > > Bastien Nocera wrote: > > > > I do use a library (or two). I've written one plugin that uses > > giflib and one that uses ImageMagick. I assumed using giflib would > > be a straighter path, but it wasn't. Firstly it only supports > > reading images > > from disk (but abydos automatically creates temporary files then > > needed > > so that didn't add any extra code at least). Secondly it doesn't do > > much more than unpacking the pixels. How to interpret what comes out > > is > > left as an exercise for the user, and requires a bit of knowledge > > about > > the GIF formats and it's quirks. So that plugin isn't built by > > default. > > ImageMagick on the other hand did much more to be of help, and > > required > > far less code to use. So shifting the responsibility to ImageMagick > > seems reasonable, I think. > > No, it really isn't: > https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html > > We want to have less CVEs, not more. I see what you mean. A few of them (although none of the more serious ones) were even related to the GIF loader specifically. But the sheer volume kind of speaks for itself otherwise. :( > > I tested them both on all the GIF images included in the gdk-pixbuf > > test suit. Both plugins mostly work, but to varying degree. The one > > based on giflib segfaults with 1_partyanimsm2.gif (because the > > allocation containing the pixels which giflib provides is less than > > the > > images width x height, I haven't yet looked deeper into it). The > > ImageMagick based plugin on the other doesn't crash at least, and > > all the invalid images are correctly classified as invalid. The > > image 1_partyanimsm2.gif still shows as garbage except the first > > frame. The image aero.gif has the frame delay set to zero for every > > frame but the > > first. I'm not sure how that should be interpreted, so I simply > > exchanged zero values for a small delay (0.02 seconds). I will read > > up > > on the GIF format and hopefully get things working better. > > > > It's available here if you want to try it out: > > http://snisurset.net/code/abydos/ > > Having looked at giflib, and knowing the author, the current plan > still is to have something based on libnsgif in the future. I guess I'll write a third GIF plugin based libnsgif then. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Mon, 2018-09-10 at 22:29 +0200, Magnus Bergman wrote: > On Mon, 10 Sep 2018 11:31:42 +0200 > Bastien Nocera wrote: > > > > I've written loader for GIF that simply wraps abydos. In lines of > > > code it's about a quarter the size of the current loader, even > > > including > > > the GIF plugin for abydos. It might even be slightly smaller with > > > the whole of abydos included in the equation. On the downside it > > > probably doesn't pass the test suite since I haven't tried it. > > > But > > > I will, and hopefully publish the whole thing in a couple of > > > days. > > > > That's unfortunately not mergeable, and unless you use a library > > for > > your GIF plugin in abydos, would just be shifting the potential > > bugs > > to the abydos code base. > > I do use a library (or two). I've written one plugin that uses giflib > and one that uses ImageMagick. I assumed using giflib would be a > straighter path, but it wasn't. Firstly it only supports reading > images > from disk (but abydos automatically creates temporary files then > needed > so that didn't add any extra code at least). Secondly it doesn't do > much more than unpacking the pixels. How to interpret what comes out > is > left as an exercise for the user, and requires a bit of knowledge > about > the GIF formats and it's quirks. So that plugin isn't built by > default. > ImageMagick on the other hand did much more to be of help, and > required > far less code to use. So shifting the responsibility to ImageMagick > seems reasonable, I think. No, it really isn't: https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html We want to have less CVEs, not more. > I tested them both on all the GIF images included in the gdk-pixbuf > test suit. Both plugins mostly work, but to varying degree. The one > based on giflib segfaults with 1_partyanimsm2.gif (because the > allocation containing the pixels which giflib provides is less than > the > images width x height, I haven't yet looked deeper into it). The > ImageMagick based plugin on the other doesn't crash at least, and all > the invalid images are correctly classified as invalid. The image > 1_partyanimsm2.gif still shows as garbage except the first frame. The > image aero.gif has the frame delay set to zero for every frame but > the > first. I'm not sure how that should be interpreted, so I simply > exchanged zero values for a small delay (0.02 seconds). I will read > up > on the GIF format and hopefully get things working better. > > It's available here if you want to try it out: > http://snisurset.net/code/abydos/ Having looked at giflib, and knowing the author, the current plan still is to have something based on libnsgif in the future. > > > > - we disable every loader by default except the ones that the > > > > core > > > > desktop needs > > > > - image viewers that rely on gdk-pixbuf ship their additional > > > > loaders > > > > in the app's Flatpak[1]. > > > > > > I don't care much for Flatpak in particular. But generalised and > > > rephrased as, leave it to the distributors to decide, I agree > > > that > > > this is absolutely the best approach. > > > > Without Flatpak, you're just removing image format support from > > image > > viewers, as many packaging guidelines in distributions forbid the > > bundling of libraries. They'd want to ship a single copy of the > > gdk- > > pixbuf loaders, and the applications wouldn't have any protection > > from > > files that the loaders would trip over. > > I'm not arguing against that. I just think it's an issue best left > entirely to distributors (including the choice between bundling and > dependencies). > > How and where to implement sandboxing is an interesting question > still. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Mon, 10 Sep 2018 07:02:23 + Debarshi Ray wrote: > On Sun, Sep 09, 2018 at 02:57:30AM +0200, Magnus Bergman wrote: > > Many fields of science deal with images of multi > > gigabyte sizes. Ideally any image viewer should be able to handle > > these too with the right plugin (probably using GEGL in that case). > > But I think the problem with large images (say 12000x12000 or so) > > is giving it to the application as a pixmap. From my own tests it > > seams it's fine at least as long as the images are no bigger than > > the screen. So if the drawing (and implicitly also scaling) is > > handed over to the loading library (which in turn might hand it > > over to the plugin), this problem can be avoided. > > Even if one does decode the entire full resolution image into a tiled > data structure (say, GeglBuffer), there's no need to create a Cairo > surface for the entire image at 1:1 zoom. All that's needed is a > surface to represent the visible area at the visible zoom. That's a > lot more manageable. Yes, exactly. Using abydos_render() that's very possible to implement by the back end. Then trying to view larger images using gdk-pixbuf, I didn't experience any particular problems then the image was at normal size or zoomed in. The problem was zooming out. So I think just creating one or two prescaled versions will get you quite far (as long as there is no problem fitting the image into memory that is). ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Mon, 10 Sep 2018 11:31:42 +0200 Bastien Nocera wrote: > > I've written loader for GIF that simply wraps abydos. In lines of > > code it's about a quarter the size of the current loader, even > > including > > the GIF plugin for abydos. It might even be slightly smaller with > > the whole of abydos included in the equation. On the downside it > > probably doesn't pass the test suite since I haven't tried it. But > > I will, and hopefully publish the whole thing in a couple of days. > > That's unfortunately not mergeable, and unless you use a library for > your GIF plugin in abydos, would just be shifting the potential bugs > to the abydos code base. I do use a library (or two). I've written one plugin that uses giflib and one that uses ImageMagick. I assumed using giflib would be a straighter path, but it wasn't. Firstly it only supports reading images from disk (but abydos automatically creates temporary files then needed so that didn't add any extra code at least). Secondly it doesn't do much more than unpacking the pixels. How to interpret what comes out is left as an exercise for the user, and requires a bit of knowledge about the GIF formats and it's quirks. So that plugin isn't built by default. ImageMagick on the other hand did much more to be of help, and required far less code to use. So shifting the responsibility to ImageMagick seems reasonable, I think. I tested them both on all the GIF images included in the gdk-pixbuf test suit. Both plugins mostly work, but to varying degree. The one based on giflib segfaults with 1_partyanimsm2.gif (because the allocation containing the pixels which giflib provides is less than the images width x height, I haven't yet looked deeper into it). The ImageMagick based plugin on the other doesn't crash at least, and all the invalid images are correctly classified as invalid. The image 1_partyanimsm2.gif still shows as garbage except the first frame. The image aero.gif has the frame delay set to zero for every frame but the first. I'm not sure how that should be interpreted, so I simply exchanged zero values for a small delay (0.02 seconds). I will read up on the GIF format and hopefully get things working better. It's available here if you want to try it out: http://snisurset.net/code/abydos/ > > > - we disable every loader by default except the ones that the core > > > desktop needs > > > - image viewers that rely on gdk-pixbuf ship their additional > > > loaders > > > in the app's Flatpak[1]. > > > > I don't care much for Flatpak in particular. But generalised and > > rephrased as, leave it to the distributors to decide, I agree that > > this is absolutely the best approach. > > Without Flatpak, you're just removing image format support from image > viewers, as many packaging guidelines in distributions forbid the > bundling of libraries. They'd want to ship a single copy of the gdk- > pixbuf loaders, and the applications wouldn't have any protection from > files that the loaders would trip over. I'm not arguing against that. I just think it's an issue best left entirely to distributors (including the choice between bundling and dependencies). How and where to implement sandboxing is an interesting question still. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Sun, 2018-09-09 at 01:23 +0200, Magnus Bergman wrote: > On Fri, 07 Sep 2018 12:51:32 +0200 > Bastien Nocera wrote: > > > > > Gegl is great for image editing. But not as much for simple > > > > viewing. > > > > > > This is debatable. If I'm viewing a 4000x4000 RGB image on a > > > hidpi > > > display I'm already pushing gdk-pixbuf and cairo to their limits > > > because of the scaling factor applied to the window — not only > > > the > > > buffer gets loaded uncompressed to allow for zooming, but the > > > image > > > viewer needs to render a CPU-scaled down copy of the image. > > > > > > Sure, for viewing a 500x400px image macro for a meme we're fine; > > > but > > > we're fine with gdk-pixbuf as well, so there's really no need to > > > change to a different image loading library. > > > > I concur, 4000x4000 would likely OOM your machine, and it's not > > really > > fixable: > > https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/30 > > > > Viewers should use GEGL, and GEGL should be taught about more > > formats. > > That still leaves many GIF files unhandled though, and I'm not sure > > we'd want apps writing their own GIF loader, seeing how complicated > > that is. > > How do you mean it's not fixable? Of course it's not fixable that > processing larger amounts of data will require more processing power. > So I think it's kind of obvious that creating thumbnails for many > large > images will take it's time. But that shouldn't need be too much of a > problem. Without looking too deeply into it, it looks like a problem > with nautilus to me. Obviously it's a bad idea to to run the > thumbnailers synchronously (which I can't imagine Nautilus does), and > it's also bad to run too many memory hungry tasks in parallel. But > that must be fixable, right? Like limiting the the total size in > bytes > of files being allowed to be processed in parallel, for example. It's not fixable without breaking the API and ABI of gdk-pixbuf. And nautilus doesn't thumbnail images itself (there's a thumbnailer that gets sandboxed by the thumbnailing code), or do it synchronously (the thumbnailers are out-of-process). > > > The animation API is pretty much a garbage fire, so it may go on > > > the chopping block as well. Of course, deprecated API will keep > > > working as well—or as badly—as it works today, so people can > > > still > > > use it. Moving pixbuf loaders to separate processes, and wrap > > > them > > > in sandboxes, would be a fairly good thing to do; it need to be > > > decided at run time, though, because there are many users of > > > GdkPixbuf that already run in a sandbox, which prevents creating > > > smaller sandboxes inside it. > > > > That'd be best left to an API that can do that natively, with the > > API > > being thought out ahead of time. > > > > In the short term, my wishlist/TODO list would be: > > - somebody writes a GIF loader that is readable, and passes the > > test > > suite. > > I've written loader for GIF that simply wraps abydos. In lines of > code it's about a quarter the size of the current loader, even > including > the GIF plugin for abydos. It might even be slightly smaller with the > whole of abydos included in the equation. On the downside it probably > doesn't pass the test suite since I haven't tried it. But I will, and > hopefully publish the whole thing in a couple of days. That's unfortunately not mergeable, and unless you use a library for your GIF plugin in abydos, would just be shifting the potential bugs to the abydos code base. > > - we disable every loader by default except the ones that the core > > desktop needs > > - image viewers that rely on gdk-pixbuf ship their additional > > loaders > > in the app's Flatpak[1]. > > I don't care much for Flatpak in particular. But generalised and > rephrased as, leave it to the distributors to decide, I agree that > this is absolutely the best approach. Without Flatpak, you're just removing image format support from image viewers, as many packaging guidelines in distributions forbid the bundling of libraries. They'd want to ship a single copy of the gdk- pixbuf loaders, and the applications wouldn't have any protection from files that the loaders would trip over. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Mon, Sep 10, 2018 at 09:40:13AM +0100, jcup...@gmail.com wrote: > I make a gtk viewer that can display large images efficiently (over > 100,000 x 100,000), linked above. I hit a few other issues: > > 1. You can't use a large ScrolledWindow and only paint the visible > area, since you can easily go over the 64k pixel limit on viewports. > You have to handle all the scrolling yourself. Yes, the View widget needs to implement GtkScrollable, and not be one massive GtkDrawingArea. > 2. You need to keep all image processing in a set of background > threads and update the display asynchronously, perhaps this is > obvious. I thought we were talking about a "simple" viewer. :) Yes, if you are doing any kind of image processing, you can't let that block the UI. But if you are just talking about zooming in and out, then a gegl_buffer_get() inside GtkWidget::draw isn't terrible. There are 8 mipmap levels, so your 100,000 x 100,000 image becomes 12500 x 12500. > 3. You have to do the double-buffering yourself as well, since it can > take a while to generate a new view and you have to update the screen > as new chunks of image are generated. You don't need double-buffering if you want to update the View as soon as new chunks are generated. That's what GIMP does. You'll need it if you want to update the entire view in one frame. Here's a standalone widget to render a GeglNode that supports animated zooms and double-buffered updates: https://gitlab.gnome.org/GNOME/gnome-photos/blob/master/src/photos-image-view.c There's also gegl-gtk, but it might be a little dusty. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Mon, 10 Sep 2018 at 08:02, Debarshi Ray wrote: > > too with the right plugin (probably using GEGL in that case). But I > > think the problem with large images (say 12000x12000 or so) is giving > > it to the application as a pixmap. From my own tests it seams it's fine > > at least as long as the images are no bigger than the screen. So if the > > drawing (and implicitly also scaling) is handed over to the loading > > library (which in turn might hand it over to the plugin), this problem > > can be avoided. > > Even if one does decode the entire full resolution image into a tiled > data structure (say, GeglBuffer), there's no need to create a Cairo > surface for the entire image at 1:1 zoom. All that's needed is a > surface to represent the visible area at the visible zoom. That's a > lot more manageable. I make a gtk viewer that can display large images efficiently (over 100,000 x 100,000), linked above. I hit a few other issues: 1. You can't use a large ScrolledWindow and only paint the visible area, since you can easily go over the 64k pixel limit on viewports. You have to handle all the scrolling yourself. 2. You need to keep all image processing in a set of background threads and update the display asynchronously, perhaps this is obvious. 3. You have to do the double-buffering yourself as well, since it can take a while to generate a new view and you have to update the screen as new chunks of image are generated. John ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Sun, Sep 09, 2018 at 02:57:30AM +0200, Magnus Bergman wrote: > Many fields of science deal with images of multi > gigabyte sizes. Ideally any image viewer should be able to handle these > too with the right plugin (probably using GEGL in that case). But I > think the problem with large images (say 12000x12000 or so) is giving > it to the application as a pixmap. From my own tests it seams it's fine > at least as long as the images are no bigger than the screen. So if the > drawing (and implicitly also scaling) is handed over to the loading > library (which in turn might hand it over to the plugin), this problem > can be avoided. Even if one does decode the entire full resolution image into a tiled data structure (say, GeglBuffer), there's no need to create a Cairo surface for the entire image at 1:1 zoom. All that's needed is a surface to represent the visible area at the visible zoom. That's a lot more manageable. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Fri, 7 Sep 2018 17:28:04 +
Debarshi Ray wrote:
> Hey Magnus,
>
> I haven't yet worked my way through the whole thread. It's pretty
> long and will take me a while longer, but I did want to mention a
> few things before the weekend draws me away from the computer.
>
> On Wed, Sep 05, 2018 at 12:02:45AM +0200, Magnus Bergman wrote:
> > Over the years it has been discussed from time to time to replace
> > gdk-pixbuf with something else[1][2]. Something was even in the
> > making (I guess over ten years ago) but it never replaced gdk-pixbuf
> > apparently. Now I don't even remember what it was called. And
> > something else called pig was proposed more recently.
>
> As Emmanuele mentioned, if you ignore the use-case of loading icons
> in GTK, and focus on image applications dealing with high resolution
> images, then GEGL [1] is the way to go. It's a GObject-based image
> processing framework that's used by GIMP and GNOME Photos, and is
> way more advanced than anything that we have in the GNOME platform.
>
> If you use GIMP 2.10 or practically any version of GNOME Photos, then
> all the image decoding and processing is happening with GEGL.
>
> A GeglBuffer is the equivalent of GdkPixbuf in GEGL. It can:
>
> (a) Be converted to and from a GdkPixbuf. That makes porting trivial
> and boosts interoperability.
>
> (b) Handle massive images that are larger than the amount of physical
> RAM available on the system. This is achieved by spliting the
> pixels into smaller tiles that get transparently swapped out of
> memory into a disk-backed cache when necessary.
>
> It can be dumbed down into a GdkPixbuf-like linear sequence of
> bytes, if need be.
>
> (c) Represent a whole horde of pixel formats, colour spaces, bit
> depths, etc. [2], which can be programmatically extended at run-time.
>
> (d) Automatically convert between pixel formats, colour spaces, bit
> depths, etc., with accelerated fast paths.
>
> (e) Be serialized into a named file, which is interesting for passing
> pixels across process boundaries.
>
> (f) Use mipmaps for scaling, rendering and processing the pixels.
>
> (g) Be used with OpenCL.
>
> (h) Do all that and is still fast. Try zooming in and out in GNOME
> Photos.
>
> Not to mention a vast array of image processing operations. If it's in
> GIMP, it's in GEGL. Interesting operations are ported from elsewhere,
> and there's a constant flow of new things under development.
GEGL is certainly impressive. Especially the ability to handle images
larger than RAM, I think. I must admit that it was some years ago that
I looked at the API and concluded that it couldn't do everything I
wanted from a general purpose image loading library (for image
viewing). Back then it could only load images from disk (but I've heard
that has changed at least). And as far as I know it still only does
raster images, not vector images. It also has no support for
animations, not to mention vector animations. Or rather, it can't (or
couldn't) distinguish between the concepts of animation frames, layers,
variants (which I explained earlier), stereoscopic images and files
that simply contains multiple images with an otherwise unspecified
relation (which I call pages in lack of a better term). I don't know
about aspect ratio correction, but I don't think GEGL has it. Some of
those things might get added to GEGL. But I doubt vector animations
will ever be considered. I will, however, take a new look at GEGL and
see if it's the best solution for at least some of the problems I want
to solve.
> > One major reason to replace gdk-pixbuf has been the long term goal
> > to phase out GDK in favour of cairo.
>
> De/encoding images and manipulating the pixels is related to rendering
> them on the screen, but aren't necessarily the same thing. It's
> important to ensure that the pixels can be efficiently drawn, but I
> wouldn't add a hard dependency on the drawing framework. For example,
> GEGL has fast paths for drawing with Cairo, but doesn't mandate it.
> It also doesn't require GTK, even though it has helper widgets for it.
>
> That's why GIMP 2.10 and GNOME Photos can use the same libgegl-0.4.so,
> even though they use different widget toolkits (GTK 2.x versus 3.x).
>
> > This is how to convert an SVG file to a PNG file.
> >
> > cairo_surface_t *surface = abydos_load("image/svg+xml",
> > "example.svg"); cairo_surface_write_to_png(surface,
> > "example.png"); cairo_surface_destroy(surface);
> >
> > This is how to convert the second frame of a GIF animation to PNG.
> >
> >abydos_t *ar = abydos_create_from_file("image/gif",
> > "example.gif"); abydos_set_frame(ar, 1);
> >cairo_surface_t *surface = abydos_get_image_surface(ar, 0);
> >abydos_destroy(ar);
> >cairo_surface_write_to_png(surface, "example.png");
> >cairo_surface_destroy(surface);
>
> I don't see any error handling. Maybe I should read the actual code.
I just intended to show
Re: An alternative to gdk-pixbuf
On Fri, 07 Sep 2018 12:51:32 +0200 Bastien Nocera wrote: > > > Gegl is great for image editing. But not as much for simple > > > viewing. > > > > This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi > > display I'm already pushing gdk-pixbuf and cairo to their limits > > because of the scaling factor applied to the window — not only the > > buffer gets loaded uncompressed to allow for zooming, but the image > > viewer needs to render a CPU-scaled down copy of the image. > > > > Sure, for viewing a 500x400px image macro for a meme we're fine; but > > we're fine with gdk-pixbuf as well, so there's really no need to > > change to a different image loading library. > > I concur, 4000x4000 would likely OOM your machine, and it's not really > fixable: > https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/30 > > Viewers should use GEGL, and GEGL should be taught about more formats. > That still leaves many GIF files unhandled though, and I'm not sure > we'd want apps writing their own GIF loader, seeing how complicated > that is. How do you mean it's not fixable? Of course it's not fixable that processing larger amounts of data will require more processing power. So I think it's kind of obvious that creating thumbnails for many large images will take it's time. But that shouldn't need be too much of a problem. Without looking too deeply into it, it looks like a problem with nautilus to me. Obviously it's a bad idea to to run the thumbnailers synchronously (which I can't imagine Nautilus does), and it's also bad to run too many memory hungry tasks in parallel. But that must be fixable, right? Like limiting the the total size in bytes of files being allowed to be processed in parallel, for example. At least now I see the problem with large images. I have no problem with 4000x4000, such images load instantly and zooms and pans smoothly. But the 11000x14000 image was a bit sluggish to zoom and pan. With Gimp it took a lot longer to load it, but panning and zooming was fast and smooth. While gdk-pixbuf is optimised for delivering one pixmap, GEGL is optimised for editing. I have some idea of how to optimise for viewing by doing a simplified form of prescaling, compared to GEGL (and hopefully outperform them both for that particular use case). Otherwise it would be possible (but probably not always optimal) to use GEGL as a backend for abydos then dealing with potentially large images. I need to investigate this. Since it's common with multi gigabyte images in many fields of science, this is something I obviously can't ignore. > > In the near future, I'll very likely deprecate most of > > > GdkPixbuf's > > > > API, except for the I/O operations; I'd also be happy to seal > > > > off most of its internals, within the ABI stability promise, to > > > > avoid leakage of internal state. > > > > > > Will the loader plugin API go away, you think? > > > > No API will ever go away: there are no plans for a gdk-pixbuf-3.0. > > The deprecations would mostly apply to API that is either long since > > been replaced by Cairo (scale/composite), or that is weirdly ad hoc, > > like saturate_and_pixelate(). Ideally, I'd like to deprecate the > > option to build gdk-pixbuf without depending on GIO to do MIME type > > sniffing, as GIO has been fixed to work on Windows and macOS, and it > > is a required dependency anyway. > > That's probably fine, though the current code in the stack is dumb as > rocks, and will try to thumbnail a JPG file with the PNG decoder if > the suffix is incorrect. The error message is also obnoxious, so > that'll need fixing before removing the content type sniffing. Then I ran gdk-pixbuf in a debugger I noticed that it ignored the file name and only used the content then determining the file type. So is there another code path which uses the filename, or is it a compile time option? I noticed that the MIME type detection code (which I think is more or less a copy paste of the reference implementation) uses the filename alone to determine the MIME type if the extension is considered reliable enough. Just as a secondary method it will use the supplied content for sniffing. It's also possible that it returns more than one MIME type and in that case gdk-pixbuf will always use the first one. This is annoying since a completely made up MIME type (like "application/x-extension-png") can be prepended to the list sometimes, so the real one gets ignored. This is apparently a feature I don't understand. > > The animation API is pretty much a garbage fire, so it may go on > > the chopping block as well. Of course, deprecated API will keep > > working as well—or as badly—as it works today, so people can still > > use it. Moving pixbuf loaders to separate processes, and wrap them > > in sandboxes, would be a fairly good thing to do; it need to be > > decided at run time, though, because there are many users of > > GdkPixbuf that already run in a sandbox, which prevents creating > >
Re: An alternative to gdk-pixbuf
On Thu, 06 Sep 2018 13:03:03 -0500 Federico Mena Quintero wrote: > On Wed, 2018-09-05 at 17:28 +0100, Emmanuele Bassi via gtk-devel-list > wrote: > > > In the near future, I'll very likely deprecate most of GdkPixbuf's > > API, except for the I/O operations; I'd also be happy to seal off > > most of its internals, within the ABI stability promise, to avoid > > leakage of internal state. > > Related to this - I just wrote a braindump on gdk-pixbuf issues and > historical baggage. I hope it's useful for people trying to clean up > this swamp: > > https://people.gnome.org/~federico/blog/my-gdk-pixbuf-braindump.html This was an interesting read. It touched many of my own ideas and concerns. Animation I actually think the concept of animations in gdk-pixbuf is okay, so it's pretty much the same in abydos, with the main difference that abydos fully supports vector animation. Apart from the frame based view of animations abydos also support continuous animations (which it seems like gdk-pixbuf also does). It's mostly the API of gdk-pixbuf that is suboptimal. Apart from GIF there are a few formats common in the art scene that supports animation. It's not just flip book animations but also palette cycling or simply one blinking color. There are also formats that show the current time (or depend on other variables that can change over time). Ideally an image viewer should update those then appropriate, so they can also be considered animations. (As you might have noticed I'm quite interested in the odd corner cases.) Mutable pixbufs The problem with mutable pixbufs is pretty much the same problem described in the documentation for abydos_get_image_surface(). The user can promise to either not modify pixels or not use the image for anything else after the modification (this is of course the fastest alternative and currently the default). Or the user can request the pixels to be copied or transferred, in which case it's safe to both modify the pixels and use the image for something else. This solution is a bit awkward, but since I think the copy/transfer scenarios are actually corner cases (which I still want to support) the main use case should remain as fast as possible, and also the one not requiring extra typing. Pluggable loaders Pluggable loaders it's something I very much appreciate since I can write my own loaders for the additional formats I care about (ten or so) and use common image viewers to view them. Otherwise I would need to write the whole viewer myself as well. The only alternatives are image viewers on original hardware (emulators may or may not cut it since the image viewers often use hardware tricks that are difficult to emulate and most of them doesn't even try very hard to correct the aspect ratio). Or try to get some strange and glitchy Windows application running in Wine, often with a suboptimal result since it doesn't support all details I care about (such as a correct aspect ratio and palette cycling). Some art groups release their own viewers, that are usually quite good. But they usually only support the subset of features their own paint applications support. One viewer to view them all would be better. But all your favourite viewers to view them all would be the best IMHO. Multi page files Multi page TIFF images and SVG images with multiple icons inside are conceptually the same to abydos. The same API could support both. Limitation of supported image formats It's clear that GTK+ doesn't need to load that many image formats. Most applications doesn't either. And it's clear that more supported formats is a bigger security risk. (But I would like to add that this isn't limited to image loading plugins, in general more software is less secure than less software). For image viewers I think the priorities are different. Most would opt for more more more. (It's not like people would opt for less games on steam just because that would reduce the attack surface, to draw a parallel.) Also, distros would probably package the more exotic loaders in different packages, not installed by default. In case applications shouldn't be allowed to read any kind of image, a filtering could be installed that only let blessed MIME types through. It could easily be added to a wrapper around abydos. Apart from the security aspect I think it's a good thing if an application can open anything you throw at it. In the word processor example you mentioned it's especially true IMHO. Personally I use Lyx, and it gives you the ability to teach it about new formats if you just tell it how to convert it to a format it already knows about. I use that a lot and find it very useful and time saving. But it's a problem if you expect others to be able to open it (which I don't, I export it to PDF instead). So for any kind of format intended for data exchange it should be limited, and clearly defined, what you can embed in it. In the end it seams to boil down to this. You can only pick two: 1)
Re: An alternative to gdk-pixbuf
On Wed, Sep 05, 2018 at 08:25:05PM +0200, Magnus Bergman wrote: > Gegl is great for image editing. But not as much for simple viewing. It > doesn't do animation People have been creating and playing videos with it: http://gegl.org/gcut.html > Also it only loads images from the > file system and is a bit bulky to use. The existing codecs aren't good, but they do support GIO URIs through GFile and streams. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Hey Magnus,
I haven't yet worked my way through the whole thread. It's pretty
long and will take me a while longer, but I did want to mention a
few things before the weekend draws me away from the computer.
On Wed, Sep 05, 2018 at 12:02:45AM +0200, Magnus Bergman wrote:
> Over the years it has been discussed from time to time to replace
> gdk-pixbuf with something else[1][2]. Something was even in the making
> (I guess over ten years ago) but it never replaced gdk-pixbuf
> apparently. Now I don't even remember what it was called. And something
> else called pig was proposed more recently.
As Emmanuele mentioned, if you ignore the use-case of loading icons
in GTK, and focus on image applications dealing with high resolution
images, then GEGL [1] is the way to go. It's a GObject-based image
processing framework that's used by GIMP and GNOME Photos, and is
way more advanced than anything that we have in the GNOME platform.
If you use GIMP 2.10 or practically any version of GNOME Photos, then
all the image decoding and processing is happening with GEGL.
A GeglBuffer is the equivalent of GdkPixbuf in GEGL. It can:
(a) Be converted to and from a GdkPixbuf. That makes porting trivial
and boosts interoperability.
(b) Handle massive images that are larger than the amount of physical
RAM available on the system. This is achieved by spliting the
pixels into smaller tiles that get transparently swapped out of
memory into a disk-backed cache when necessary.
It can be dumbed down into a GdkPixbuf-like linear sequence of
bytes, if need be.
(c) Represent a whole horde of pixel formats, colour spaces, bit depths,
etc. [2], which can be programmatically extended at run-time.
(d) Automatically convert between pixel formats, colour spaces, bit
depths, etc., with accelerated fast paths.
(e) Be serialized into a named file, which is interesting for passing
pixels across process boundaries.
(f) Use mipmaps for scaling, rendering and processing the pixels.
(g) Be used with OpenCL.
(h) Do all that and is still fast. Try zooming in and out in GNOME
Photos.
Not to mention a vast array of image processing operations. If it's in
GIMP, it's in GEGL. Interesting operations are ported from elsewhere,
and there's a constant flow of new things under development.
> One major reason to replace gdk-pixbuf has been the long term goal to
> phase out GDK in favour of cairo.
De/encoding images and manipulating the pixels is related to rendering
them on the screen, but aren't necessarily the same thing. It's
important to ensure that the pixels can be efficiently drawn, but I
wouldn't add a hard dependency on the drawing framework. For example,
GEGL has fast paths for drawing with Cairo, but doesn't mandate it.
It also doesn't require GTK, even though it has helper widgets for it.
That's why GIMP 2.10 and GNOME Photos can use the same libgegl-0.4.so,
even though they use different widget toolkits (GTK 2.x versus 3.x).
> This is how to convert an SVG file to a PNG file.
>
> cairo_surface_t *surface = abydos_load("image/svg+xml",
> "example.svg"); cairo_surface_write_to_png(surface, "example.png");
> cairo_surface_destroy(surface);
>
> This is how to convert the second frame of a GIF animation to PNG.
>
>abydos_t *ar = abydos_create_from_file("image/gif", "example.gif");
>abydos_set_frame(ar, 1);
>cairo_surface_t *surface = abydos_get_image_surface(ar, 0);
>abydos_destroy(ar);
>cairo_surface_write_to_png(surface, "example.png");
>cairo_surface_destroy(surface);
I don't see any error handling. Maybe I should read the actual code.
Either way, as Emmanuele mentioned, it's really important to have
support for GIO facilities like GCancellable, GFile, streams,
GAsyncResult-based asynchronous APIs, etc..
If you look closely at GEGL, you'll see that while it already has
codecs, they are missing these requirements. It's an artifact of
GIMP historically having it's own in-house codecs.
However, that's changing. I have a work-in-progress GNOME Photos
branch [3] that prototypes a better codec API for GeglBuffer.
See:
*
https://gitlab.gnome.org/GNOME/gnome-photos/blob/wip/rishi/buffer-decoder/src/photos-gegl-buffer-io.h
*
https://gitlab.gnome.org/GNOME/gnome-photos/blob/wip/rishi/buffer-decoder/src/photos-gegl-buffer-loader.h
One obvious thing is that it looks almost identical to the corresponding
GdkPixbuf API. That's an explicit goal to make porting trivial.
Of course, one could use the GdkPixbuf codecs for I/O and then covert
to and from a GeglBuffer. But then you wouldn't be able to handle RAWs
or higher bit-depth PNGs and will likely run into problems with your
80 megapixel images [4] - things that a good image application will
care about. :)
> I've been in contact with a two creators of image viewers
Which ones? Just curious. :)
> What it all comes down to is that using a bigger variety of software
> creates a bigger attack surface. Viewing a lot
Re: An alternative to gdk-pixbuf
On Thu, 2018-09-06 at 11:39 +0100, Emmanuele Bassi via gtk-devel-list wrote: > On Wed, 5 Sep 2018 at 19:25, Magnus Bergman < > magnus.berg...@snisurset.net> wrote: > > On Wed, 5 Sep 2018 17:28:22 +0100 > > Emmanuele Bassi wrote: > > > > > We're phasing out Cairo in favour of the CSS rendering model, > > > implemented on top of OpenGL and Vulkan, as it's the API that > > most > > > closely matches the requirements of GTK. > > > > I'm not sure I quite understand what you are saying. What does this > > mean for image loading if terms of actual implementation? What > > would > > ideally happen then GTK+ needs to load an image (because the > > application called gtk_image_new_from_file() for example)? > > > > We're still using GdkPixbuf, and for the 4.0 API series we still have > GdkPixbuf types exposed in the GTK API. > > For future API series (5.x, 6.x, …) we may revisit this, with the > option of moving icon loading into GTK itself. This uncertainty is the reason why I didn't work on moving the thumbnailing code somewhere else, as it's bound to be made irrelevant sooner rather than later. > > Gegl is great for image editing. But not as much for simple > > viewing. > > This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi > display I'm already pushing gdk-pixbuf and cairo to their limits > because of the scaling factor applied to the window — not only the > buffer gets loaded uncompressed to allow for zooming, but the image > viewer needs to render a CPU-scaled down copy of the image. > > Sure, for viewing a 500x400px image macro for a meme we're fine; but > we're fine with gdk-pixbuf as well, so there's really no need to > change to a different image loading library. I concur, 4000x4000 would likely OOM your machine, and it's not really fixable: https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/30 Viewers should use GEGL, and GEGL should be taught about more formats. That still leaves many GIF files unhandled though, and I'm not sure we'd want apps writing their own GIF loader, seeing how complicated that is. > In the near future, I'll very likely deprecate most of > > GdkPixbuf's > > > API, except for the I/O operations; I'd also be happy to seal off > > > most of its internals, within the ABI stability promise, to avoid > > > leakage of internal state. > > > > Will the loader plugin API go away, you think? > > No API will ever go away: there are no plans for a gdk-pixbuf-3.0. > The deprecations would mostly apply to API that is either long since > been replaced by Cairo (scale/composite), or that is weirdly ad hoc, > like saturate_and_pixelate(). Ideally, I'd like to deprecate the > option to build gdk-pixbuf without depending on GIO to do MIME type > sniffing, as GIO has been fixed to work on Windows and macOS, and it > is a required dependency anyway. That's probably fine, though the current code in the stack is dumb as rocks, and will try to thumbnail a JPG file with the PNG decoder if the suffix is incorrect. The error message is also obnoxious, so that'll need fixing before removing the content type sniffing. > The animation API is pretty much a garbage fire, so it may go on the > chopping block as well. Of course, deprecated API will keep working > as well—or as badly—as it works today, so people can still use it. > Moving pixbuf loaders to separate processes, and wrap them in > sandboxes, would be a fairly good thing to do; it need to be decided > at run time, though, because there are many users of GdkPixbuf that > already run in a sandbox, which prevents creating smaller sandboxes > inside it. That'd be best left to an API that can do that natively, with the API being thought out ahead of time. In the short term, my wishlist/TODO list would be: - somebody writes a GIF loader that is readable, and passes the test suite. - we disable every loader by default except the ones that the core desktop needs - image viewers that rely on gdk-pixbuf ship their additional loaders in the app's Flatpak[1]. That would already reduce the attack surface for gdk-pixbuf. On a sidenote, I also use gdk-pixbuf as a pixel-perfect drawing API for thumbnailers: https://gitlab.gnome.org/Archive/gnome-nds-thumbnailer/blob/master/gnome-nds-thumbnailer.c#L72 [1]: I recently did this to add webp comics support to the evince Flatpak: https://gitlab.gnome.org/GNOME/evince/merge_requests/50 ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Thu, 6 Sep 2018 11:39:59 +0100 Emmanuele Bassi wrote: > On Wed, 5 Sep 2018 at 19:25, Magnus Bergman > wrote: > > > On Wed, 5 Sep 2018 17:28:22 +0100 > > Emmanuele Bassi wrote: > > > > > We're phasing out Cairo in favour of the CSS rendering model, > > > implemented on top of OpenGL and Vulkan, as it's the API that most > > > closely matches the requirements of GTK. > > > > I'm not sure I quite understand what you are saying. What does this > > mean for image loading if terms of actual implementation? What would > > ideally happen then GTK+ needs to load an image (because the > > application called gtk_image_new_from_file() for example)? > > > > > We're still using GdkPixbuf, and for the 4.0 API series we still have > GdkPixbuf types exposed in the GTK API. > > For future API series (5.x, 6.x, …) we may revisit this, with the > option of moving icon loading into GTK itself. Okay, sound reasonable. > > > Cairo is still used as a fallback mechanism — both when running on > > > platforms without support for OpenGL or Vulkan, and in the interim > > > period while the GL/Vulkan rendering pipelines inside GTK don't > > > support a CSS feature. Additionally, you may still use Cairo for > > > 2D high quality rendering, for printing and for custom drawing. > > > > But then it comes to printing and custom drawing I'm on my own then > > it comes to loading the images? Which is okey of course since > > gdk-pixbuf is kind of a separate library already. But isn't it a > > good thing to share common image loading code? > > > > Not really; icons—especially ones that are identified via icon theme > names—are a very narrow subset of "all the possible images and > formats in human history". > > > Gegl is great for image editing. But not as much for simple viewing. > > This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi > display I'm already pushing gdk-pixbuf and cairo to their limits > because of the scaling factor applied to the window — not only the > buffer gets loaded uncompressed to allow for zooming, but the image > viewer needs to render a CPU-scaled down copy of the image. I often deal with images of those sizes and I haven't experienced any problems with this myself (sure it's not as fast as viewing small images). But I will look into it. I though cairo was capable of utilising the rendering back end for it's scaling. I don't see why cairo NEEDS to use CPU-scaling, so maybe it can be fixed. > > It doesn't do animation > > Animated formats are a dying breed, and they have all but killed by > VP9 and new, web-friendly video formats. Social platforms will take a > GIF and turn it into a video transparently. > > Additionally, GTK 4.x already has new API to render videos and other > frame-based media sources through GStreamer. I must agree this sounds like a good decision. Even though I don't think animated GIFs will die any time soon, it's not really anything GTK+ should need to pay special attention to. From my perspective, being interested in the art scene, animation might not be very common, but formats supporting animation certainly are. It's not only in the form of a series of frames (like GIF), it's also palette cycling, or simply one blinking color I include in the concept of animation. I've also seen image formats that shows the current time. Those, I think, should ideally be displayed with the time updating. So that's also a kind of animation. Does GStreamer support vector animations, by the way? > > > From the perspective of a consumer of GdkPixbuf, the only thing > > > that > > > an image loading library should do is, pretty literally, load > > > images. No scaling, no compositing, no rendering. Saving to a > > > file may be interesting — but that opens the whole transcoding > > > can of worms, so maybe it should just be a debugging feature. > > > Ideally, I/O operations should happen in a separate thread, and > > > possible use multi-threading to chunk out the work on multiple > > > cores; it definitely needs to integrate with GIO, as it's a > > > modern API that well maps to GUIs. > > > > Yes, I very much agree with this. Except I think it has to do > > rendering of some sort. If an image contains a triangle, it has to > > be converted to a function call that renders it (using > > OpenGL/Vulcan/cairo) somehow. > > That's up to the toolkit, as it's going to be the one deciding how to > render the rest of the UI. There's no way for you to know, from the > outside, how to efficiently render anything. > > That's why it's much, much better to get a memory buffer with the > contents of the image, and let the toolkit pass it to the GPU. > > Vector formats are different, but that's why we have API like Cairo or > Skia; I'm also waiting for web browsers to implement SVG rendering on > the GPU where possible, with the help of new primitives from the > GL/Vulkan implementation. That means one code path for images that are certainly raster based and another
Re: An alternative to gdk-pixbuf
On Wed, 2018-09-05 at 17:28 +0100, Emmanuele Bassi via gtk-devel-list wrote: > In the near future, I'll very likely deprecate most of GdkPixbuf's > API, except for the I/O operations; I'd also be happy to seal off > most of its internals, within the ABI stability promise, to avoid > leakage of internal state. Related to this - I just wrote a braindump on gdk-pixbuf issues and historical baggage. I hope it's useful for people trying to clean up this swamp: https://people.gnome.org/~federico/blog/my-gdk-pixbuf-braindump.html Magnus - thanks for showing us Abydos. The parts of the API that intersect with gdk-pixbuf's look okay. I think you may find this useful: https://developer.gnome.org/programming-guidelines/stable/index.html.en I think your API could improve with some of the material there around memory management and error reporting. Internally, the code needs checks in all the malloc()-like functions; some of the arithmetic (especially in assertions) needs to avoid overflows. I couldn't find any tests. The SVG plug-in needs to handle error results from librsvg. The Magick plug-in for PNG and JPEG has no error handling; also you may want to call cairo_surface_mark_dirty() when you are done writing pixels to it. The netpbm plugin does a bunch of system calls and doesn't check for errors. The parts of your API that deal with pagination / variants / animation frames - I'm not sure if a gdk-pixbuf replacement would need them, but I'm coming from the viewpoint of my braindump above. Federico ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Thu, 6 Sep 2018 at 11:40, Emmanuele Bassi via gtk-devel-list wrote: > On Wed, 5 Sep 2018 at 19:25, Magnus Bergman > wrote: >> Gegl is great for image editing. But not as much for simple viewing. > > This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi display > I'm already pushing gdk-pixbuf and cairo to their limits because of the > scaling factor applied to the window — not only the buffer gets loaded > uncompressed to allow for zooming, but the image viewer needs to render a > CPU-scaled down copy of the image. It doesn't have to be gegl, of course, you could use any image processing library to load and scale down the images. I made a tiny image viewer based on libvips: https://github.com/jcupitt/vipsdisp-tiny It's 300 lines, but does multi-threaded, asynchronous painting of many image formats. It can display gigapixel images on very modest hardware, and it should be quick. There's a more complete image viewer here: https://github.com/jcupitt/vipsdisp That adds most of the usual navigation stuff, though it's not quite done, I must get back to it. The image display part is a widget you could easily cut out and paste into other applications. It's in C, but libvips is a GObject-based library, so it'd be easy to write in any language. Here it is displaying a 18k x 8k 16-bit TIFF: http://www.rollthepotato.net/~john/summer.png That image takes about 8s to load on this laptop. Here it is displaying a 120k x 100k pixel ndpi slide image: http://www.rollthepotato.net/~john/slide.png That image loads instantly, since ndpi supports random access and vipsdisp can just load the parts it needs to paint the screen. John ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Wed, 5 Sep 2018 at 19:25, Magnus Bergman wrote: > On Wed, 5 Sep 2018 17:28:22 +0100 > Emmanuele Bassi wrote: > > > We're phasing out Cairo in favour of the CSS rendering model, > > implemented on top of OpenGL and Vulkan, as it's the API that most > > closely matches the requirements of GTK. > > I'm not sure I quite understand what you are saying. What does this > mean for image loading if terms of actual implementation? What would > ideally happen then GTK+ needs to load an image (because the > application called gtk_image_new_from_file() for example)? > > We're still using GdkPixbuf, and for the 4.0 API series we still have GdkPixbuf types exposed in the GTK API. For future API series (5.x, 6.x, …) we may revisit this, with the option of moving icon loading into GTK itself. > > Cairo is still used as a fallback mechanism — both when running on > > platforms without support for OpenGL or Vulkan, and in the interim > > period while the GL/Vulkan rendering pipelines inside GTK don't > > support a CSS feature. Additionally, you may still use Cairo for 2D > > high quality rendering, for printing and for custom drawing. > > But then it comes to printing and custom drawing I'm on my own then it > comes to loading the images? Which is okey of course since gdk-pixbuf > is kind of a separate library already. But isn't it a good thing to > share common image loading code? > Not really; icons—especially ones that are identified via icon theme names—are a very narrow subset of "all the possible images and formats in human history". Gegl is great for image editing. But not as much for simple viewing. This is debatable. If I'm viewing a 4000x4000 RGB image on a hidpi display I'm already pushing gdk-pixbuf and cairo to their limits because of the scaling factor applied to the window — not only the buffer gets loaded uncompressed to allow for zooming, but the image viewer needs to render a CPU-scaled down copy of the image. Sure, for viewing a 500x400px image macro for a meme we're fine; but we're fine with gdk-pixbuf as well, so there's really no need to change to a different image loading library. > It doesn't do animation Animated formats are a dying breed, and they have all but killed by VP9 and new, web-friendly video formats. Social platforms will take a GIF and turn it into a video transparently. Additionally, GTK 4.x already has new API to render videos and other frame-based media sources through GStreamer. > From the perspective of a consumer of GdkPixbuf, the only thing that > > an image loading library should do is, pretty literally, load images. > > No scaling, no compositing, no rendering. Saving to a file may be > > interesting — but that opens the whole transcoding can of worms, so > > maybe it should just be a debugging feature. Ideally, I/O operations > > should happen in a separate thread, and possible use multi-threading > > to chunk out the work on multiple cores; it definitely needs to > > integrate with GIO, as it's a modern API that well maps to GUIs. > > Yes, I very much agree with this. Except I think it has to do rendering > of some sort. If an image contains a triangle, it has to be converted to > a function call that renders it (using OpenGL/Vulcan/cairo) somehow. > That's up to the toolkit, as it's going to be the one deciding how to render the rest of the UI. There's no way for you to know, from the outside, how to efficiently render anything. That's why it's much, much better to get a memory buffer with the contents of the image, and let the toolkit pass it to the GPU. Vector formats are different, but that's why we have API like Cairo or Skia; I'm also waiting for web browsers to implement SVG rendering on the GPU where possible, with the help of new primitives from the GL/Vulkan implementation. > In the near future, I'll very likely deprecate most of GdkPixbuf's > > API, except for the I/O operations; I'd also be happy to seal off > > most of its internals, within the ABI stability promise, to avoid > > leakage of internal state. > > Will the loader plugin API go away, you think? > No API will ever go away: there are no plans for a gdk-pixbuf-3.0. The deprecations would mostly apply to API that is either long since been replaced by Cairo (scale/composite), or that is weirdly ad hoc, like saturate_and_pixelate(). Ideally, I'd like to deprecate the option to build gdk-pixbuf without depending on GIO to do MIME type sniffing, as GIO has been fixed to work on Windows and macOS, and it is a required dependency anyway. The animation API is pretty much a garbage fire, so it may go on the chopping block as well. Of course, deprecated API will keep working as well—or as badly—as it works today, so people can still use it. Moving pixbuf loaders to separate processes, and wrap them in sandboxes, would be a fairly good thing to do; it need to be decided at run time, though, because there are many users of GdkPixbuf that already run in a sandbox, which prevents creating
Re: An alternative to gdk-pixbuf
On 09/05/2018 06:57 PM, Nicolas Dufresne wrote: > I've replied to a comment about sandboxing image loading to prevent > possible crash buffer overflow from happening. You are now focusing on > an optimization. I believe you should start a new thread. It's relevant because it is the same as the bubblewrap case. You pass an FD to the mmap'able file across the IPC boundary. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Le mercredi 05 septembre 2018 à 18:43 -0700, Christian Hergert a écrit : > On 09/05/2018 06:18 PM, Nicolas Dufresne wrote: > > Is there any benchmark that would justify this added complexity ? Also, > > there will be more context switch, so cache misses will take more time > > then just loading the icon directly. > > Just because you've decoded into a non-shareable page of memory doesn't > mean it wont swap and be trivialized into the same as the cached case > but with everything else worse. > > Also, read-only caches can have the benefit of shoving the index table > into the head of the file and embedding it in the inode ensuring that > even your lookup table is fast and share-able. I've replied to a comment about sandboxing image loading to prevent possible crash buffer overflow from happening. You are now focusing on an optimization. I believe you should start a new thread. Nicolas ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 09/05/2018 06:18 PM, Nicolas Dufresne wrote: > Is there any benchmark that would justify this added complexity ? Also, > there will be more context switch, so cache misses will take more time > then just loading the icon directly. Just because you've decoded into a non-shareable page of memory doesn't mean it wont swap and be trivialized into the same as the cached case but with everything else worse. Also, read-only caches can have the benefit of shoving the index table into the head of the file and embedding it in the inode ensuring that even your lookup table is fast and share-able. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Wed, 5 Sep 2018 17:47:57 -0400 Ray Strode wrote: > hi, > > On Tue, Sep 4, 2018, 6:19 PM Magnus Bergman > wrote: > > > Over the years it has been discussed from time to time to replace > > gdk-pixbuf with something else[1][2]. > > [...] > > > I finally took some time to design an > > image loading library on top of cairo > > [...] > > > abydos, which at least > > suits my needs. And also some needs mentioned in this list over the > > years. First I thought it could suit the needs of GTK+ as well. > > So one thing i think a modern image library needs is sandboxing. the > loader should be contained with bubblewrap and it should just give > back a chunk of pixels in shared memory as a memfd or similar. > > then if it crashes it doesnt take down the app. if its exploited the > most the attacker can do is make the caller show the wrong image > (which could still be bad of course) Yes, I though about that and there is an example implementation in the netpbm plugin of abydos that takes this approach. It currently doesn't drop privileges, but that can easily be added. But it only works with pixmaps. I'm not sure how to deal with vector graphics this way. I will take a look at bubblewrap and think this over. But if opening images is considered dangerous, opening files in general must be considered equally dangerous. And if third party image loaders are considered dangerous, all third party code must be considered equally dangerous. I think it would be better with some kind of framework that can separate applications into run-time modules with different privileges. So that all kind of data processing happens in contexts without access to the file system for example. Then the whole image loader (in this case better described as an image decoder) could be placed in the same isolated context together with the rest of the data processing stuff. It would be kind of cool if every application always did this. But I think this technique is primarily needed in especially sensitive situations (like handling complex and potentially malicious data like from the internet), not as much for loading local images in general. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Le mercredi 05 septembre 2018 à 17:49 -0700, Christian Hergert a écrit : > On 09/05/2018 05:03 PM, Nicolas Dufresne wrote: > > > > For foreign image, yes, but for system icons, that's just an > > overhead. > > System icons should be using mmap'able caches that avoid any runtime > overhead and allows read-only page-sharing between processes. Is there any benchmark that would justify this added complexity ? Also, there will be more context switch, so cache misses will take more time then just loading the icon directly. > ___ > gtk-devel-list mailing list > gtk-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/gtk-devel-list ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 09/05/2018 05:03 PM, Nicolas Dufresne wrote: > > For foreign image, yes, but for system icons, that's just an overhead. System icons should be using mmap'able caches that avoid any runtime overhead and allows read-only page-sharing between processes. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Le mer. 5 sept. 2018 17:48, Ray Strode via gtk-devel-list < gtk-devel-list@gnome.org> a écrit : > hi, > > On Tue, Sep 4, 2018, 6:19 PM Magnus Bergman > wrote: > >> Over the years it has been discussed from time to time to replace >> gdk-pixbuf with something else[1][2]. > > [...] > >> I finally took some time to design an >> image loading library on top of cairo > > [...] > >> abydos, which at least >> suits my needs. And also some needs mentioned in this list over the >> years. First I thought it could suit the needs of GTK+ as well. > > So one thing i think a modern image library needs is sandboxing. the > loader should be contained with bubblewrap and it should just give back a > chunk of pixels in shared memory as a memfd or similar. > > then if it crashes it doesnt take down the app. if its exploited the most > the attacker can do is make the caller show the wrong image (which could > still be bad of course) > For foreign image, yes, but for system icons, that's just an overhead. > --Ray > ___ > gtk-devel-list mailing list > gtk-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/gtk-devel-list > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
hi, On Tue, Sep 4, 2018, 6:19 PM Magnus Bergman wrote: > Over the years it has been discussed from time to time to replace > gdk-pixbuf with something else[1][2]. [...] > I finally took some time to design an > image loading library on top of cairo [...] > abydos, which at least > suits my needs. And also some needs mentioned in this list over the > years. First I thought it could suit the needs of GTK+ as well. So one thing i think a modern image library needs is sandboxing. the loader should be contained with bubblewrap and it should just give back a chunk of pixels in shared memory as a memfd or similar. then if it crashes it doesnt take down the app. if its exploited the most the attacker can do is make the caller show the wrong image (which could still be bad of course) --Ray ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Wed, 5 Sep 2018 17:28:22 +0100 Emmanuele Bassi wrote: > Hi; > > On Tue, 4 Sep 2018 at 23:19, Magnus Bergman > wrote: > > > Over the years it has been discussed from time to time to replace > > gdk-pixbuf with something else[1][2]. Something was even in the > > making (I guess over ten years ago) but it never replaced gdk-pixbuf > > apparently. Now I don't even remember what it was called. And > > something else called pig was proposed more recently. > > > > One major reason to replace gdk-pixbuf has been the long term goal > > to phase out GDK in favour of cairo. And some people (including me) > > has been displeased with the gdk-pixbuf API for more subtle > > reasons. It has also been brough up that it lacks some features > > (which I guess are hard to implement as an afterthought). I finally > > took some time to design an image loading library on top of cairo > > called abydos, which at least suits my needs. And also some needs > > mentioned in this list over the years. First I thought it could > > suit the needs of GTK+ as well. But I just learned that even cairo > > is now considered "legacy technology" by the GTK+ developers[3], so > > I guess cairo is also being phased out now. But in favour of what? > > > We're phasing out Cairo in favour of the CSS rendering model, > implemented on top of OpenGL and Vulkan, as it's the API that most > closely matches the requirements of GTK. I'm not sure I quite understand what you are saying. What does this mean for image loading if terms of actual implementation? What would ideally happen then GTK+ needs to load an image (because the application called gtk_image_new_from_file() for example)? Will GTK+ have a limited set of hard coded loaders which can render images directly using both OpenGL and Vulcan? So one custom SVG loader for OpenGL and one custom SVG loader for Vulcan. And likewise for every other supported image format (I guess at least PNG, JPEG, GIF, WEBP and Windows Icon). > Cairo is still used as a fallback mechanism — both when running on > platforms without support for OpenGL or Vulkan, and in the interim > period while the GL/Vulkan rendering pipelines inside GTK don't > support a CSS feature. Additionally, you may still use Cairo for 2D > high quality rendering, for printing and for custom drawing. But then it comes to printing and custom drawing I'm on my own then it comes to loading the images? Which is okey of course since gdk-pixbuf is kind of a separate library already. But isn't it a good thing to share common image loading code? > Internally, GTK only cares about GdkPixbuf in as much as it provides > us with a way to load graphic assets like icons, which are typically > in a very limited amount of formats. As far as we're concerned, image > data coming from GdkPixbuf and Cairo gets loaded into memory buffers > that get submitted to the GPU almost immediately, and all > transformations and rendering happen using shaders, on the GPU. > > Anything bigger than an icon should probably be loaded and displayed > through Gegl, the same library used by the GIMP; this is especially > true if you're planning to process the image using filters. Images, > these days, are pretty big buffers — and so are displays; using a > simple linear buffer like GdkPixbuf's does not scale. I'm not convinced that you should need to use one image loader for loading small images and another for loading full scale images (such as ordinary photos). Then it comes to really large images (many times the RAM size) I can agree that something else might be needed. Gegl is great for image editing. But not as much for simple viewing. It doesn't do animation and doesn't support vector graphics (apart from converting them into pixmaps). Also it only loads images from the file system and is a bit bulky to use. It simply doesn't fit the needs for a simple image viewer. > From the perspective of a consumer of GdkPixbuf, the only thing that > an image loading library should do is, pretty literally, load images. > No scaling, no compositing, no rendering. Saving to a file may be > interesting — but that opens the whole transcoding can of worms, so > maybe it should just be a debugging feature. Ideally, I/O operations > should happen in a separate thread, and possible use multi-threading > to chunk out the work on multiple cores; it definitely needs to > integrate with GIO, as it's a modern API that well maps to GUIs. Yes, I very much agree with this. Except I think it has to do rendering of some sort. If an image contains a triangle, it has to be converted to a function call that renders it (using OpenGL/Vulcan/cairo) somehow. Just handing over a set of geometrical objects hasn't taken you very far from the original coded data, basically just converted it into another vector data format. Spreading the work over multiple threads I mostly consider an implementation detail. Or do you think it needs to be considered then designing an API for image loading? And if
Re: An alternative to gdk-pixbuf
Hi; On Tue, 4 Sep 2018 at 23:19, Magnus Bergman wrote: > Over the years it has been discussed from time to time to replace > gdk-pixbuf with something else[1][2]. Something was even in the making > (I guess over ten years ago) but it never replaced gdk-pixbuf > apparently. Now I don't even remember what it was called. And something > else called pig was proposed more recently. > > One major reason to replace gdk-pixbuf has been the long term goal to > phase out GDK in favour of cairo. And some people (including me) has > been displeased with the gdk-pixbuf API for more subtle reasons. It has > also been brough up that it lacks some features (which I guess are hard > to implement as an afterthought). I finally took some time to design an > image loading library on top of cairo called abydos, which at least > suits my needs. And also some needs mentioned in this list over the > years. First I thought it could suit the needs of GTK+ as well. But I > just learned that even cairo is now considered "legacy technology" by > the GTK+ developers[3], so I guess cairo is also being phased out now. > But in favour of what? We're phasing out Cairo in favour of the CSS rendering model, implemented on top of OpenGL and Vulkan, as it's the API that most closely matches the requirements of GTK. Cairo is still used as a fallback mechanism — both when running on platforms without support for OpenGL or Vulkan, and in the interim period while the GL/Vulkan rendering pipelines inside GTK don't support a CSS feature. Additionally, you may still use Cairo for 2D high quality rendering, for printing and for custom drawing. Internally, GTK only cares about GdkPixbuf in as much as it provides us with a way to load graphic assets like icons, which are typically in a very limited amount of formats. As far as we're concerned, image data coming from GdkPixbuf and Cairo gets loaded into memory buffers that get submitted to the GPU almost immediately, and all transformations and rendering happen using shaders, on the GPU. Anything bigger than an icon should probably be loaded and displayed through Gegl, the same library used by the GIMP; this is especially true if you're planning to process the image using filters. Images, these days, are pretty big buffers — and so are displays; using a simple linear buffer like GdkPixbuf's does not scale. >From the perspective of a consumer of GdkPixbuf, the only thing that an image loading library should do is, pretty literally, load images. No scaling, no compositing, no rendering. Saving to a file may be interesting — but that opens the whole transcoding can of worms, so maybe it should just be a debugging feature. Ideally, I/O operations should happen in a separate thread, and possible use multi-threading to chunk out the work on multiple cores; it definitely needs to integrate with GIO, as it's a modern API that well maps to GUIs. In the near future, I'll very likely deprecate most of GdkPixbuf's API, except for the I/O operations; I'd also be happy to seal off most of its internals, within the ABI stability promise, to avoid leakage of internal state. Ciao, Emmanuele. -- https://www.bassi.io [@] ebassi [@gmail.com] ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
An alternative to gdk-pixbuf
Over the years it has been discussed from time to time to replace
gdk-pixbuf with something else[1][2]. Something was even in the making
(I guess over ten years ago) but it never replaced gdk-pixbuf
apparently. Now I don't even remember what it was called. And something
else called pig was proposed more recently.
One major reason to replace gdk-pixbuf has been the long term goal to
phase out GDK in favour of cairo. And some people (including me) has
been displeased with the gdk-pixbuf API for more subtle reasons. It has
also been brough up that it lacks some features (which I guess are hard
to implement as an afterthought). I finally took some time to design an
image loading library on top of cairo called abydos, which at least
suits my needs. And also some needs mentioned in this list over the
years. First I thought it could suit the needs of GTK+ as well. But I
just learned that even cairo is now considered "legacy technology" by
the GTK+ developers[3], so I guess cairo is also being phased out now.
But in favour of what? Maybe the basic design of my API could still be
of some interest. And perhaps I could even port my code over to whatever
is the next hot drawing library.
It's mainly a proof of concept right now. But it works well enough to be
tested. It doesn't have many loaders, but I assume it will be a trivial
task to port them over from gdk-pixbuf since the plugin API:s are quite
similar. The API for handling variants (see below) is work in
progress and not even implemented yet. And I plan to add support for
stereoscopic images as well.
The main features of abydos
* It's easy to use, especially for the common use cases.
* Depends only on cairo. (The main library that is, loaders may of
course have other dependancies.)
* Selects what loader to use based on MIME types. (But does nothing to
determine the MIME types by itself.)
* Full support for vector graphics. (Scaling and other transformations
can be done using cairo.)
* Supports animations, both frame based and continous. (But does
nothing to measure clock time or schedule events by itself.) Where is
also no intention to support proper video (streaming) formats.
* Supports layers. As in toggling layers on and off. Just delivering
each layer as a separate image might not be feasible since a layer
can be more like a filter operation than a pixmap.
* Supports multi page images. This means image files that contains more
than one image. Which for example TIFF files can.
* Supports multi variant images. This is somewhat like multi page images
except that it is known that the images are all (differently sized)
variants of the same image. It could be embedded thumbnails (which
could be intelligently cropped and not just scaled down versions of
the full size image). Or it could be icons with different details
level for different sizes. Or it could be mipmap textures.
* Deals with non square pixels. By default it scales images to the
correct aspect ratio, but can optionally deliver them pixel by pixel
as well.
What it lacks in comparison to gdk-pixbuf
* It (currently) lacks support for saving images. Of course images can
be saved to PDF, PNG and SVG using cairo. I would like to add for
some more general purpose animation format to complement those (MNG
or APNG probably). As far as I know the image saving functionality of
gdk-pixbuf isn't heavily used. At least not for something other than
PNG.
* It lacks support for indexed colors, since cairo lacks that support.
(So this is more of a question about what cairo lacks in comparison
to GDK.)
* It lacks support for determining the MIME type of files. This is a
hard problem to solve and I think it's better solved elsewhere. Using
the shared MIME database for sniffing goes a long way, and is surely
enough for the needs of GTK+. But not all types can be detected this
way. Using libextractor (with special plugins) goes a bit further.
* It doesn't use GObject. But adding GObject bindings would be possible
I guess.
Some examples
This is how to convert an SVG file to a PNG file.
cairo_surface_t *surface = abydos_load("image/svg+xml",
"example.svg"); cairo_surface_write_to_png(surface, "example.png");
cairo_surface_destroy(surface);
This is how to convert the second frame of a GIF animation to PNG.
abydos_t *ar = abydos_create_from_file("image/gif", "example.gif");
abydos_set_frame(ar, 1);
cairo_surface_t *surface = abydos_get_image_surface(ar, 0);
abydos_destroy(ar);
cairo_surface_write_to_png(surface, "example.png");
cairo_surface_destroy(surface);
This is how to display a GIF animation to something cairo can draw on.
Of course some kind of event scheduling should be used insted of a
delay in the real world.
abydos_t *ar = abydos_create_from_file("image/gif", "example.gif");
double last_timestamp = now();
for(;;) {
abydos_render(ar, cr, 0);
delay(abydos_get_delay(ar));
