Re: An alternative to gdk-pixbuf
On 09/05/2018 06:57 PM, Nicolas Dufresne wrote: > I've replied to a comment about sandboxing image loading to prevent > possible crash buffer overflow from happening. You are now focusing on > an optimization. I believe you should start a new thread. It's relevant because it is the same as the bubblewrap case. You pass an FD to the mmap'able file across the IPC boundary. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Le mercredi 05 septembre 2018 à 18:43 -0700, Christian Hergert a écrit : > On 09/05/2018 06:18 PM, Nicolas Dufresne wrote: > > Is there any benchmark that would justify this added complexity ? Also, > > there will be more context switch, so cache misses will take more time > > then just loading the icon directly. > > Just because you've decoded into a non-shareable page of memory doesn't > mean it wont swap and be trivialized into the same as the cached case > but with everything else worse. > > Also, read-only caches can have the benefit of shoving the index table > into the head of the file and embedding it in the inode ensuring that > even your lookup table is fast and share-able. I've replied to a comment about sandboxing image loading to prevent possible crash buffer overflow from happening. You are now focusing on an optimization. I believe you should start a new thread. Nicolas ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 09/05/2018 06:18 PM, Nicolas Dufresne wrote: > Is there any benchmark that would justify this added complexity ? Also, > there will be more context switch, so cache misses will take more time > then just loading the icon directly. Just because you've decoded into a non-shareable page of memory doesn't mean it wont swap and be trivialized into the same as the cached case but with everything else worse. Also, read-only caches can have the benefit of shoving the index table into the head of the file and embedding it in the inode ensuring that even your lookup table is fast and share-able. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Wed, 5 Sep 2018 17:47:57 -0400 Ray Strode wrote: > hi, > > On Tue, Sep 4, 2018, 6:19 PM Magnus Bergman > wrote: > > > Over the years it has been discussed from time to time to replace > > gdk-pixbuf with something else[1][2]. > > [...] > > > I finally took some time to design an > > image loading library on top of cairo > > [...] > > > abydos, which at least > > suits my needs. And also some needs mentioned in this list over the > > years. First I thought it could suit the needs of GTK+ as well. > > So one thing i think a modern image library needs is sandboxing. the > loader should be contained with bubblewrap and it should just give > back a chunk of pixels in shared memory as a memfd or similar. > > then if it crashes it doesnt take down the app. if its exploited the > most the attacker can do is make the caller show the wrong image > (which could still be bad of course) Yes, I though about that and there is an example implementation in the netpbm plugin of abydos that takes this approach. It currently doesn't drop privileges, but that can easily be added. But it only works with pixmaps. I'm not sure how to deal with vector graphics this way. I will take a look at bubblewrap and think this over. But if opening images is considered dangerous, opening files in general must be considered equally dangerous. And if third party image loaders are considered dangerous, all third party code must be considered equally dangerous. I think it would be better with some kind of framework that can separate applications into run-time modules with different privileges. So that all kind of data processing happens in contexts without access to the file system for example. Then the whole image loader (in this case better described as an image decoder) could be placed in the same isolated context together with the rest of the data processing stuff. It would be kind of cool if every application always did this. But I think this technique is primarily needed in especially sensitive situations (like handling complex and potentially malicious data like from the internet), not as much for loading local images in general. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Le mercredi 05 septembre 2018 à 17:49 -0700, Christian Hergert a écrit : > On 09/05/2018 05:03 PM, Nicolas Dufresne wrote: > > > > For foreign image, yes, but for system icons, that's just an > > overhead. > > System icons should be using mmap'able caches that avoid any runtime > overhead and allows read-only page-sharing between processes. Is there any benchmark that would justify this added complexity ? Also, there will be more context switch, so cache misses will take more time then just loading the icon directly. > ___ > gtk-devel-list mailing list > gtk-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/gtk-devel-list ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On 09/05/2018 05:03 PM, Nicolas Dufresne wrote: > > For foreign image, yes, but for system icons, that's just an overhead. System icons should be using mmap'able caches that avoid any runtime overhead and allows read-only page-sharing between processes. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
Le mer. 5 sept. 2018 17:48, Ray Strode via gtk-devel-list < gtk-devel-list@gnome.org> a écrit : > hi, > > On Tue, Sep 4, 2018, 6:19 PM Magnus Bergman > wrote: > >> Over the years it has been discussed from time to time to replace >> gdk-pixbuf with something else[1][2]. > > [...] > >> I finally took some time to design an >> image loading library on top of cairo > > [...] > >> abydos, which at least >> suits my needs. And also some needs mentioned in this list over the >> years. First I thought it could suit the needs of GTK+ as well. > > So one thing i think a modern image library needs is sandboxing. the > loader should be contained with bubblewrap and it should just give back a > chunk of pixels in shared memory as a memfd or similar. > > then if it crashes it doesnt take down the app. if its exploited the most > the attacker can do is make the caller show the wrong image (which could > still be bad of course) > For foreign image, yes, but for system icons, that's just an overhead. > --Ray > ___ > gtk-devel-list mailing list > gtk-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/gtk-devel-list > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
hi, On Tue, Sep 4, 2018, 6:19 PM Magnus Bergman wrote: > Over the years it has been discussed from time to time to replace > gdk-pixbuf with something else[1][2]. [...] > I finally took some time to design an > image loading library on top of cairo [...] > abydos, which at least > suits my needs. And also some needs mentioned in this list over the > years. First I thought it could suit the needs of GTK+ as well. So one thing i think a modern image library needs is sandboxing. the loader should be contained with bubblewrap and it should just give back a chunk of pixels in shared memory as a memfd or similar. then if it crashes it doesnt take down the app. if its exploited the most the attacker can do is make the caller show the wrong image (which could still be bad of course) --Ray ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: An alternative to gdk-pixbuf
On Wed, 5 Sep 2018 17:28:22 +0100 Emmanuele Bassi wrote: > Hi; > > On Tue, 4 Sep 2018 at 23:19, Magnus Bergman > wrote: > > > Over the years it has been discussed from time to time to replace > > gdk-pixbuf with something else[1][2]. Something was even in the > > making (I guess over ten years ago) but it never replaced gdk-pixbuf > > apparently. Now I don't even remember what it was called. And > > something else called pig was proposed more recently. > > > > One major reason to replace gdk-pixbuf has been the long term goal > > to phase out GDK in favour of cairo. And some people (including me) > > has been displeased with the gdk-pixbuf API for more subtle > > reasons. It has also been brough up that it lacks some features > > (which I guess are hard to implement as an afterthought). I finally > > took some time to design an image loading library on top of cairo > > called abydos, which at least suits my needs. And also some needs > > mentioned in this list over the years. First I thought it could > > suit the needs of GTK+ as well. But I just learned that even cairo > > is now considered "legacy technology" by the GTK+ developers[3], so > > I guess cairo is also being phased out now. But in favour of what? > > > We're phasing out Cairo in favour of the CSS rendering model, > implemented on top of OpenGL and Vulkan, as it's the API that most > closely matches the requirements of GTK. I'm not sure I quite understand what you are saying. What does this mean for image loading if terms of actual implementation? What would ideally happen then GTK+ needs to load an image (because the application called gtk_image_new_from_file() for example)? Will GTK+ have a limited set of hard coded loaders which can render images directly using both OpenGL and Vulcan? So one custom SVG loader for OpenGL and one custom SVG loader for Vulcan. And likewise for every other supported image format (I guess at least PNG, JPEG, GIF, WEBP and Windows Icon). > Cairo is still used as a fallback mechanism — both when running on > platforms without support for OpenGL or Vulkan, and in the interim > period while the GL/Vulkan rendering pipelines inside GTK don't > support a CSS feature. Additionally, you may still use Cairo for 2D > high quality rendering, for printing and for custom drawing. But then it comes to printing and custom drawing I'm on my own then it comes to loading the images? Which is okey of course since gdk-pixbuf is kind of a separate library already. But isn't it a good thing to share common image loading code? > Internally, GTK only cares about GdkPixbuf in as much as it provides > us with a way to load graphic assets like icons, which are typically > in a very limited amount of formats. As far as we're concerned, image > data coming from GdkPixbuf and Cairo gets loaded into memory buffers > that get submitted to the GPU almost immediately, and all > transformations and rendering happen using shaders, on the GPU. > > Anything bigger than an icon should probably be loaded and displayed > through Gegl, the same library used by the GIMP; this is especially > true if you're planning to process the image using filters. Images, > these days, are pretty big buffers — and so are displays; using a > simple linear buffer like GdkPixbuf's does not scale. I'm not convinced that you should need to use one image loader for loading small images and another for loading full scale images (such as ordinary photos). Then it comes to really large images (many times the RAM size) I can agree that something else might be needed. Gegl is great for image editing. But not as much for simple viewing. It doesn't do animation and doesn't support vector graphics (apart from converting them into pixmaps). Also it only loads images from the file system and is a bit bulky to use. It simply doesn't fit the needs for a simple image viewer. > From the perspective of a consumer of GdkPixbuf, the only thing that > an image loading library should do is, pretty literally, load images. > No scaling, no compositing, no rendering. Saving to a file may be > interesting — but that opens the whole transcoding can of worms, so > maybe it should just be a debugging feature. Ideally, I/O operations > should happen in a separate thread, and possible use multi-threading > to chunk out the work on multiple cores; it definitely needs to > integrate with GIO, as it's a modern API that well maps to GUIs. Yes, I very much agree with this. Except I think it has to do rendering of some sort. If an image contains a triangle, it has to be converted to a function call that renders it (using OpenGL/Vulcan/cairo) somehow. Just handing over a set of geometrical objects hasn't taken you very far from the original coded data, basically just converted it into another vector data format. Spreading the work over multiple threads I mostly consider an implementation detail. Or do you think it needs to be considered then designing an API for image loading? And if yo
Re: An alternative to gdk-pixbuf
Hi; On Tue, 4 Sep 2018 at 23:19, Magnus Bergman wrote: > Over the years it has been discussed from time to time to replace > gdk-pixbuf with something else[1][2]. Something was even in the making > (I guess over ten years ago) but it never replaced gdk-pixbuf > apparently. Now I don't even remember what it was called. And something > else called pig was proposed more recently. > > One major reason to replace gdk-pixbuf has been the long term goal to > phase out GDK in favour of cairo. And some people (including me) has > been displeased with the gdk-pixbuf API for more subtle reasons. It has > also been brough up that it lacks some features (which I guess are hard > to implement as an afterthought). I finally took some time to design an > image loading library on top of cairo called abydos, which at least > suits my needs. And also some needs mentioned in this list over the > years. First I thought it could suit the needs of GTK+ as well. But I > just learned that even cairo is now considered "legacy technology" by > the GTK+ developers[3], so I guess cairo is also being phased out now. > But in favour of what? We're phasing out Cairo in favour of the CSS rendering model, implemented on top of OpenGL and Vulkan, as it's the API that most closely matches the requirements of GTK. Cairo is still used as a fallback mechanism — both when running on platforms without support for OpenGL or Vulkan, and in the interim period while the GL/Vulkan rendering pipelines inside GTK don't support a CSS feature. Additionally, you may still use Cairo for 2D high quality rendering, for printing and for custom drawing. Internally, GTK only cares about GdkPixbuf in as much as it provides us with a way to load graphic assets like icons, which are typically in a very limited amount of formats. As far as we're concerned, image data coming from GdkPixbuf and Cairo gets loaded into memory buffers that get submitted to the GPU almost immediately, and all transformations and rendering happen using shaders, on the GPU. Anything bigger than an icon should probably be loaded and displayed through Gegl, the same library used by the GIMP; this is especially true if you're planning to process the image using filters. Images, these days, are pretty big buffers — and so are displays; using a simple linear buffer like GdkPixbuf's does not scale. >From the perspective of a consumer of GdkPixbuf, the only thing that an image loading library should do is, pretty literally, load images. No scaling, no compositing, no rendering. Saving to a file may be interesting — but that opens the whole transcoding can of worms, so maybe it should just be a debugging feature. Ideally, I/O operations should happen in a separate thread, and possible use multi-threading to chunk out the work on multiple cores; it definitely needs to integrate with GIO, as it's a modern API that well maps to GUIs. In the near future, I'll very likely deprecate most of GdkPixbuf's API, except for the I/O operations; I'd also be happy to seal off most of its internals, within the ABI stability promise, to avoid leakage of internal state. Ciao, Emmanuele. -- https://www.bassi.io [@] ebassi [@gmail.com] ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
