GLib 2.20 released
GLib 2.20.0 is now available for download at: ftp://ftp.gtk.org/pub/glib/2.20 http://download.gnome.org/sources/glib/2.20 glib-2.20.0.tar.bz2 md5sum: 05188e560dbd8bddd568e24ce10dd003 glib-2.20.0.tar.gzmd5sum: 81a6151c9591378af31d78e7eb304e71 GLib is the low-level core library that forms the basis for projects such as GTK+ and GNOME. It provides data structure handling for C, portability wrappers, and interfaces for such runtime functionality as an event loop, threads, dynamic loading, and an object system. GLib 2.20 is a stable release adding an incremental improvement in functionality over GLib 2.18 while maintaining binary and source compatibility. Major new features include: * Optimizations of GHashTable and GMarkup * Support for in-place decoding of base64 strings * When launching applications, always pass fuse file:// uris when possible, and let gio convert such uris back to gio uris. This can be turned off using the X-GIO-NoFuse desktop file key. * GIO has a concept of 'shadowed' mounts For more details and lists of fixed bugs, see the announcements of the 2.19.x development releases: http://mail.gnome.org/archives/gtk-devel-list/2008-October/msg00115.html http://mail.gnome.org/archives/gtk-devel-list/2008-December/msg0.html http://mail.gnome.org/archives/gtk-devel-list/2008-December/msg1.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg6.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg00088.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg7.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg00069.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg00070.html http://mail.gnome.org/archives/gtk-devel-list/2009-March/msg5.html More information about GLib is available at: http://www.gtk.org/ The GLib Team: Hans Breuer, Matthias Clasen, Tim Janik, Tor Lillqvist, Manish Singh, Owen Taylor, and Sebastian Wilhelmi Thanks to all contributors, including Alexander Larsson Andreas Köhler Andrew Feren Andrzej Zaborowski Anis Elleuch Armin Burgmeier A. Walton Axel von Bertoldi Bastien Nocera Behdad Esfahbod Björn Lindqvist Christian Dywan Christian Persch Christophe Fergeau Colin Walters Cosimo Cecchi Daniel Marjamäki Dan Williams Dan Winship David Zeuthen Dimitri Vorbiev Enrico Tröger Fabian Groffen Federico Mena Quintero Grahame Bowland Guillaume Desmottes Hans Petter Jansson Hrvoje Niksic Jakub Jelinek Jens Georg Jody Goldberg Joseph Pingenot Kazuki Iwamoto Kjartan Maraas Krysztof Kosiński Leonardo Ferreira Fontenelle Loïc Minier Maciej Piechotka Markus Duft Mart Raudsepp Mathias Hasselmann Matthias Drochner Matt Johnston Michael Natterer Mike Kestner Murray Cumming Nelson Benítez León Nicolas Joseph Olivier Sessink Øystein Johansen Paolo Borelli Pascal Terjan Patryk Zawadzki Paul Pogonyshev Peter Kjellerstedt Philip Withnall Ryan Lortie Sebastian Dröge Sebastien Bacher Stefan Kost Stef Walter Sven Herzberg Takeshi Aihana Tommi Komulainen Will Thompson March 13, 2009 Matthias Clasen ___ gtk-app-devel-list mailing list gtk-app-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-app-devel-list
GTK+ 2.16.0 released
GTK+ 2.16.0 is now available for download at: http://download.gnome.org/sources/gtk+/2.16/ ftp://ftp.gtk.org/pub/gtk/2.16/ gtk+-2.16.0.tar.bz2 md5sum: 139528802794287427fd4d18875b5cf5 gtk+-2.16.0.tar.gzmd5sum: 2e926e526850d9cf62883c6d909b4351 Release notes are available at: http://www.gtk.org/notes/gtk-2.16.0-notes.txt What is GTK+ GTK+ is a multi-platform toolkit for creating graphical user interfaces. Offering a complete set of widgets, GTK+ is suitable for projects ranging from small one-off tools to complete application suites. GTK+ has been designed from the ground up to support a range of languages, not only C/C++. Using GTK+ from languages such as Perl and Python (especially in combination with the Glade GUI builder) provides an effective method of rapid application development. GTK+ is free software and part of the GNU Project. However, the licensing terms for GTK+, the GNU LGPL, allow it to be used by all developers, including those developing proprietary software, without any license fees or royalties. GTK+ is the only 100% free-of-cost open source industrial-strength GUI toolkit available today. Since its origins as the toolkit for the GNU Image Manipulation Program (GIMP), GTK+ has been used in a wide range of software. Notably, GTK+ is the foundation of the GNOME desktop; GTK+ 2.14 will be incorporated into version 2.24 of the GNOME desktop. What's new in GTK+ 2.16 === GTK+ 2.16 adds new functionality while maintaining source and binary compatibility with GTK+ 2.14. Highlights of new features and improvements in GTK+ 2.16 are: Noteworthy new APIs - GtkOrientable is a new interface implemented by all widgets that have horizontal and vertical variants - GtkActivatable is a new interface that must be implemented by widgets that are used as action proxies Changes in GtkEntry - GtkEntry can show icons at either side of the entry, which can be made clickable, drag sources, etc. - GtkEntry can show progress information behind the text. - GTK+ picks the best available placeholder character for password entries unless one is explicitly set. - GTK+ displays a Caps Lock warning in password entries. - Input methods can now be selected per-widget with the im-module property. Changes in GtkScale - GtkScale can now display annotated marks for certain values. Changes in GtkStatusIcon - GTK+ uses an extension of the tray icon specification to negotiate RGBA support. This is also supported by the GNOME panel. - GtkStatusIcon now supports scroll events, middle clicks and rich tooltips if the platform allows it. Changes in file chooser - The file chooser can optionally show file sizes. - Volumes are mounted when necessary. - GTK+ remembers the file chooser's window size across invocations. - Non-local uris can be entered in the location entry. Changes in printing support - Page rendering can be deferred to a thread. Internationalization support - Keyboard shortcut handling has been changed, to help with a longstanding complaint about the way in which GTK+ handles multiple keyboard layouts. GTK+ now only uses keys from groups other than the current group if they are not present in the current group. GDK changes - GdkKeymap emits a state-changed signal when the Caps Lock state changes. - There is a predefined type for blank cursors, GDK_BLANK_CURSOR. For more details and lists of fixed bugs, see the announcements for the 2.15.x development releases: http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg0.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg00115.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg00151.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg8.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg00073.html Where to get more information about GTK+ Information about GTK+ including links to documentation can be found at: http://www.gtk.org/ An installation guide for GTK+ 2.x is found at: http://developer.gnome.org/doc/API/2.0/gtk/gtk-building.html Common questions: http://developer.gnome.org/doc/API/2.0/gtk/gtk-question-index.html http://www.gtk.org/faq/ Contributing GTK+ is a large project and relies on voluntary contributions. We are actively searching for new contributors in various areas and invite everyone to help project development. If you are willing to participate, please subscribe to the project mailing lists to offer your help and read over our list of vacant project tasks: http://live.gnome.org/GtkTasks The GTK+ Team: Jonathan Blandford, Hans Breuer, Kristian Rietveld, Matthias Clasen, Tim Janik, Tor Lillqvist, Federico Mena Quintero, Søren Sandmann, Manish Singh, and Owen Taylor Thanks to everybody else who has contributed to this release: Akira Tagoh, Alberto Ruiz, Alexander Larsson,
Re: fsync in glib/gio
On Thu, 2009-03-12 at 21:27 +, Michael Meeks wrote: On Thu, 2009-03-12 at 21:11 +0100, Alexander Larsson wrote: With all the recent yahoo about ext4 data loss and fsync I felt I had to look at glib and make sure we're doing this right. Hmm; is this not just a database guy ? ;-) presumably if -all- file I/O should be synchronous, the kernel would do this for us ? If you want to you can make all i/o sync by mounting it as such. But thats of course really slow. Generally the gio file write operations are used for saving files, and people sort of expect that when save returns the file is ok on disk. And to make matters worse, its perfectly ok for a filesystem (ext4, xfs and btrfs do this atm) to reorder the metadata and the data writes such that writing to a temp file, renaming over target file and then crashing can end up with an empty file. This happens if metadata was saved but not the new file contents, and the window for this is about a minute, so its not a small race condition. So, you save and the system hangs 10 seconds later. What do you expect? Ideally the new file, less problematic the old file. But, if you're left with *no* version of the file I'd be pretty pissed off. Attached is a patch that makes sure we fsync before closing in the gio file saving code and in g_file_set_contents(). Isn't it the case that with ext3 and below fsync is an impossibly expensive operation that gums up the whole system - by taking some obscure kernel lock on some other piece of somethingummy and causes everything to grind to a halt, your audio to skip, and instant hair loss ? ;-) With the data=ordered setting in ext3 (the default), any fsync will result in all dirty data being flushed, not just the data in that file. This can be pretty expensive if there is a lot of outstanding I/O. However, this is only a problem if such an operation happens often, and file saving is just not that common. And if something constantly is saving something that is a problem for multiple other reasons too and should be fixed. Of course, not all file writes are saves. For example, it could be nautilus copying 1 files. This is why I added the ASYNC_WRITE flag and used it in the file copy case. I believe they fixed this for ext4, which is nice for them; but ... for everyone else ? What data-loss case are we really trying to protect against ? of course, if you hard yank the power, bad things can happen; but how often does that occur ? It occurs often enought that there were several people in the ubuntu ext4-eats-my-data bug that had it happen to them multiple times. AFAIR we spent some cycles in evolution recently to reduce the ridiculous number of fsync's that sqlite was injecting into each transaction to make the message store perform reasonably and not grind the whole system to a halt ;-) at 10ms per fsync, that makes some sense. The sqlite case is slightly different, basically same as the firefox case: http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/ Basically, once you're syncing the database regularly we're talking about constantly syncing, not just syncing when you're finished saving. So, the problem is much worse. I think in the firefox case it synced for every key you pressed in the awesome bar. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Hi Alex, On Fri, 2009-03-13 at 08:38 +0100, Alexander Larsson wrote: If you want to you can make all i/o sync by mounting it as such. But thats of course really slow. Generally the gio file write operations are used for saving files, and people sort of expect that when save returns the file is ok on disk. Sure - which is basically what ext3 provides with it's default ordered mode right ? metadata hits the disk after data - ok, so all pending data writes were flushed as well as a side-effect(?) but ... ;-) And to make matters worse, its perfectly ok for a filesystem (ext4, xfs and btrfs do this atm) to reorder the metadata and the data writes such that writing to a temp file, renaming over target file and then crashing can end up with an empty file. This happens if metadata was saved but not the new file contents, and the window for this is about a minute, so its not a small race condition. Sure, sure - and I guess this is why they had to make 'fsync' not suck for these filing systems. So, you save and the system hangs 10 seconds later. What do you expect? Ideally the new file, less problematic the old file. But, if you're left with *no* version of the file I'd be pretty pissed off. Yep; of course this is not good. With the data=ordered setting in ext3 (the default), any fsync will result in all dirty data being flushed, not just the data in that file. This can be pretty expensive if there is a lot of outstanding I/O. Sure - on ext3 it's a 'sync()' call effectively, with the added bonus that this has a terrible effect on other applications trying to use the I/O subsystem to eg. read audio to play your mp3, or swap, or allocate memory, or ... ;-) Calling 'fsync' regularly on ext3 will bring your system to a grinding halt, quite regularly, cf. my comments on hair-loss etc. Lots of our users will be on these systems, and last I checked [ did you poke your kernel guys on this ? ] this was not recommended. However, this is only a problem if such an operation happens often, and file saving is just not that common. Sure; although amusingly, if it happens often ~enough (say every few hundred ms) - then we almost move the OS into a semi-synchronous writes mode, and have somewhat less to write each time, and so don't suffer multi-second glitches in the I/O subsystem ;-) And if something constantly is saving something that is a problem for multiple other reasons too and should be fixed. Oh - well, of course there is regular autosave, and setting of gconf settings [ we set ~200+ keys on login amazingly ;-], then of course the IM infrastructure will want to make sure your IM logs are *really* on-disk each time you get a message, the E-mail client for every mail message ;-) pretty soon if everyone calls 'fsync' we end up in a fairly bad place (IMHO). Of course, not all file writes are saves. For example, it could be nautilus copying 1 files. This is why I added the ASYNC_WRITE flag and used it in the file copy case. I guess. It occurs often enought that there were several people in the ubuntu ext4-eats-my-data bug that had it happen to them multiple times. Nasty indeed, and this is the only solution. I suppose the nutshell of my concern is this: * can we in some (evil or otherwise) way avoid hurting desktop performance, interactivity and playback for all ext2/3 systems, and still keep ext4 users happy ? :-) Of course, perhaps I'm just way off / out of date wrt. my dislike of fsync, let me hunt down some kernely people for a sane 2nd opinion. Regards, Michael. -- michael.me...@novell.com , Pseudo Engineer, itinerant idiot ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Alexander Larsson wrote: Attached is a patch that makes sure we fsync before closing in the gio file saving code and in g_file_set_contents(). Wouldn't fdatasync be sufficient in most case ? ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On Fri, 2009-03-13 at 11:37 +0100, Steve Frécinaux wrote: Alexander Larsson wrote: Attached is a patch that makes sure we fsync before closing in the gio file saving code and in g_file_set_contents(). Wouldn't fdatasync be sufficient in most case ? In practical gio use there really is no difference. As soon as the size of the file changes the metadata (st_size) changes, so a fdatasync() implies fsync(). The difference is only apparent when you do in place changing of a file, which would happen e.g. in a database file. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Am Freitag, den 13.03.2009, 12:18 +0100 schrieb Sven Neumann: Hi, On Fri, 2009-03-13 at 08:38 +0100, Alexander Larsson wrote: If you want to you can make all i/o sync by mounting it as such. But thats of course really slow. Generally the gio file write operations are used for saving files, and people sort of expect that when save returns the file is ok on disk. Do they?? Doing file I/O asynchronously is a feature, in particular for laptop users. It improves I/O performance and it saves power. Of course it's a risk and may result in data loss under certain rare circumstances. But it's a risk that people are willing to take. Please do not ruin this by implicitly enforcing fsync. I think you don't understand the problem. Other file systems but ext3 in order=data mode are that brain dead and broken, that they lose __both__ the old and new document on power loss! This is __not__ acceptable, in no way. Maybe the time kernel hackers will realize some day, that they lost any sense for real world applications and over-optimized their file systems for write performance benchmarks. Well, but until this happens we have to suffer from fsync(). Really, loosing both versions of files really isn't an option. Ciao, Mathias -- Mathias Hasselmann mathias.hasselm...@gmx.de Personal Blog: http://taschenorakel.de/mathias/ Openismus GmbH: http://www.openismus.com/ ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
I think I am in line with what Michael is saying here: there is a non-trivial risk that littering fsync all over the place will badly affect existing systems. The ext4 attitude is interesting, btw. They are saying that POSIX allows this behaviour so it's your problem. But when the gcc people say The C standard allows this or that, then the kernel people question the sanity of gcc developers several generations back. F*** POSIX allows this! A program that does open-write-close- rename should not be left with an empty file in case something goes wrong. The old file, or the new file. Anything else is insane and by extension the kernel developers and their ancestors. The world is full of crappy hardware, drivers, electricity suppliers, cable-pulling clumsy people, etc. They are all out to get you, so there is no reason why the kernel should hand them your data too. Morten ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
GLib 2.20 released
GLib 2.20.0 is now available for download at: ftp://ftp.gtk.org/pub/glib/2.20 http://download.gnome.org/sources/glib/2.20 glib-2.20.0.tar.bz2 md5sum: 05188e560dbd8bddd568e24ce10dd003 glib-2.20.0.tar.gzmd5sum: 81a6151c9591378af31d78e7eb304e71 GLib is the low-level core library that forms the basis for projects such as GTK+ and GNOME. It provides data structure handling for C, portability wrappers, and interfaces for such runtime functionality as an event loop, threads, dynamic loading, and an object system. GLib 2.20 is a stable release adding an incremental improvement in functionality over GLib 2.18 while maintaining binary and source compatibility. Major new features include: * Optimizations of GHashTable and GMarkup * Support for in-place decoding of base64 strings * When launching applications, always pass fuse file:// uris when possible, and let gio convert such uris back to gio uris. This can be turned off using the X-GIO-NoFuse desktop file key. * GIO has a concept of 'shadowed' mounts For more details and lists of fixed bugs, see the announcements of the 2.19.x development releases: http://mail.gnome.org/archives/gtk-devel-list/2008-October/msg00115.html http://mail.gnome.org/archives/gtk-devel-list/2008-December/msg0.html http://mail.gnome.org/archives/gtk-devel-list/2008-December/msg1.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg6.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg00088.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg7.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg00069.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg00070.html http://mail.gnome.org/archives/gtk-devel-list/2009-March/msg5.html More information about GLib is available at: http://www.gtk.org/ The GLib Team: Hans Breuer, Matthias Clasen, Tim Janik, Tor Lillqvist, Manish Singh, Owen Taylor, and Sebastian Wilhelmi Thanks to all contributors, including Alexander Larsson Andreas Köhler Andrew Feren Andrzej Zaborowski Anis Elleuch Armin Burgmeier A. Walton Axel von Bertoldi Bastien Nocera Behdad Esfahbod Björn Lindqvist Christian Dywan Christian Persch Christophe Fergeau Colin Walters Cosimo Cecchi Daniel Marjamäki Dan Williams Dan Winship David Zeuthen Dimitri Vorbiev Enrico Tröger Fabian Groffen Federico Mena Quintero Grahame Bowland Guillaume Desmottes Hans Petter Jansson Hrvoje Niksic Jakub Jelinek Jens Georg Jody Goldberg Joseph Pingenot Kazuki Iwamoto Kjartan Maraas Krysztof Kosiński Leonardo Ferreira Fontenelle Loïc Minier Maciej Piechotka Markus Duft Mart Raudsepp Mathias Hasselmann Matthias Drochner Matt Johnston Michael Natterer Mike Kestner Murray Cumming Nelson Benítez León Nicolas Joseph Olivier Sessink Øystein Johansen Paolo Borelli Pascal Terjan Patryk Zawadzki Paul Pogonyshev Peter Kjellerstedt Philip Withnall Ryan Lortie Sebastian Dröge Sebastien Bacher Stefan Kost Stef Walter Sven Herzberg Takeshi Aihana Tommi Komulainen Will Thompson March 13, 2009 Matthias Clasen ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Hi, On Fri, 2009-03-13 at 14:11 +0100, Mathias Hasselmann wrote: I think you don't understand the problem. That might very well be the case. I had a look at the presentation that Alex linked to in the initial post in this thread. But I would have preferred a document that doesn't look at the issue from a database developer point of view. Other file systems but ext3 in order=data mode are that brain dead and broken, that they lose __both__ the old and new document on power loss! This is __not__ acceptable, in no way. But ext3 is what everyone uses. And as far as I understand the next generation Linux file-system btrfs is going to provide similar functionality: http://btrfs.wiki.kernel.org/index.php/FAQ#Does_Btrfs_have_data.3Dordered_mode_like_Ext3.3F It seems wrong to work around broken file-systems on the application level. That only takes away pressure from the file-system developers to address the problem properly. Sven ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On Fri, Mar 13, 2009 at 12:37 PM, Steve Frécinaux nudr...@gmail.com wrote: Alexander Larsson wrote: Attached is a patch that makes sure we fsync before closing in the gio file saving code and in g_file_set_contents(). Wouldn't fdatasync be sufficient in most case ? Narrowed reply: not all OS'es have fdatasync(). Therefore, you will need at least fallback to fsync(). General reply: People, are you sure that trashing of all OS'es with all filesystems by general-purpose library is a good way to workaround _application_ problem on the one concrete OS plus one concrete FS combination? IMHO, it is indded application level problem. On databases -- yes, we need to save any and every data at any cost. But it is extreme case and people who writting DB servers do syncs anyway independly on OS and FS, just because of nature of application. But when paranoia-mode begins drive anything anywere on the desktop... Unsure that it is good. If follow it, then all partitions should me remounted with -o sync mode and all other modes should be declared obsolete and removed. Unsure that it is good (again)... -- Andrew W. Nosenko andrew.w.nose...@gmail.com ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On 03/13/2009 09:15 AM, Morten Welinder wrote: I think I am in line with what Michael is saying here: there is a non-trivial risk that littering fsync all over the place will badly affect existing systems. The ext4 attitude is interesting, btw. They are saying that POSIX allows this behaviour so it's your problem. But when the gcc people say The C standard allows this or that, then the kernel people question the sanity of gcc developers several generations back. F*** POSIX allows this! A program that does open-write-close- rename should not be left with an empty file in case something goes wrong. The old file, or the new file. Anything else is insane and by extension the kernel developers and their ancestors. Very well said. Great way to start a Friday :). Keep it coming guys! Cheers, behdad The world is full of crappy hardware, drivers, electricity suppliers, cable-pulling clumsy people, etc. They are all out to get you, so there is no reason why the kernel should hand them your data too. Morten ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
GTK+ 2.16.0 released
GTK+ 2.16.0 is now available for download at: http://download.gnome.org/sources/gtk+/2.16/ ftp://ftp.gtk.org/pub/gtk/2.16/ gtk+-2.16.0.tar.bz2 md5sum: 139528802794287427fd4d18875b5cf5 gtk+-2.16.0.tar.gzmd5sum: 2e926e526850d9cf62883c6d909b4351 Release notes are available at: http://www.gtk.org/notes/gtk-2.16.0-notes.txt What is GTK+ GTK+ is a multi-platform toolkit for creating graphical user interfaces. Offering a complete set of widgets, GTK+ is suitable for projects ranging from small one-off tools to complete application suites. GTK+ has been designed from the ground up to support a range of languages, not only C/C++. Using GTK+ from languages such as Perl and Python (especially in combination with the Glade GUI builder) provides an effective method of rapid application development. GTK+ is free software and part of the GNU Project. However, the licensing terms for GTK+, the GNU LGPL, allow it to be used by all developers, including those developing proprietary software, without any license fees or royalties. GTK+ is the only 100% free-of-cost open source industrial-strength GUI toolkit available today. Since its origins as the toolkit for the GNU Image Manipulation Program (GIMP), GTK+ has been used in a wide range of software. Notably, GTK+ is the foundation of the GNOME desktop; GTK+ 2.14 will be incorporated into version 2.24 of the GNOME desktop. What's new in GTK+ 2.16 === GTK+ 2.16 adds new functionality while maintaining source and binary compatibility with GTK+ 2.14. Highlights of new features and improvements in GTK+ 2.16 are: Noteworthy new APIs - GtkOrientable is a new interface implemented by all widgets that have horizontal and vertical variants - GtkActivatable is a new interface that must be implemented by widgets that are used as action proxies Changes in GtkEntry - GtkEntry can show icons at either side of the entry, which can be made clickable, drag sources, etc. - GtkEntry can show progress information behind the text. - GTK+ picks the best available placeholder character for password entries unless one is explicitly set. - GTK+ displays a Caps Lock warning in password entries. - Input methods can now be selected per-widget with the im-module property. Changes in GtkScale - GtkScale can now display annotated marks for certain values. Changes in GtkStatusIcon - GTK+ uses an extension of the tray icon specification to negotiate RGBA support. This is also supported by the GNOME panel. - GtkStatusIcon now supports scroll events, middle clicks and rich tooltips if the platform allows it. Changes in file chooser - The file chooser can optionally show file sizes. - Volumes are mounted when necessary. - GTK+ remembers the file chooser's window size across invocations. - Non-local uris can be entered in the location entry. Changes in printing support - Page rendering can be deferred to a thread. Internationalization support - Keyboard shortcut handling has been changed, to help with a longstanding complaint about the way in which GTK+ handles multiple keyboard layouts. GTK+ now only uses keys from groups other than the current group if they are not present in the current group. GDK changes - GdkKeymap emits a state-changed signal when the Caps Lock state changes. - There is a predefined type for blank cursors, GDK_BLANK_CURSOR. For more details and lists of fixed bugs, see the announcements for the 2.15.x development releases: http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg0.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg00115.html http://mail.gnome.org/archives/gtk-devel-list/2009-January/msg00151.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg8.html http://mail.gnome.org/archives/gtk-devel-list/2009-February/msg00073.html Where to get more information about GTK+ Information about GTK+ including links to documentation can be found at: http://www.gtk.org/ An installation guide for GTK+ 2.x is found at: http://developer.gnome.org/doc/API/2.0/gtk/gtk-building.html Common questions: http://developer.gnome.org/doc/API/2.0/gtk/gtk-question-index.html http://www.gtk.org/faq/ Contributing GTK+ is a large project and relies on voluntary contributions. We are actively searching for new contributors in various areas and invite everyone to help project development. If you are willing to participate, please subscribe to the project mailing lists to offer your help and read over our list of vacant project tasks: http://live.gnome.org/GtkTasks The GTK+ Team: Jonathan Blandford, Hans Breuer, Kristian Rietveld, Matthias Clasen, Tim Janik, Tor Lillqvist, Federico Mena Quintero, Søren Sandmann, Manish Singh, and Owen Taylor Thanks to everybody else who has contributed to this release: Akira Tagoh, Alberto Ruiz, Alexander Larsson,
Re: fsync in glib/gio
On Fri, 2009-03-13 at 14:35 +0100, Sven Neumann wrote: Hi, On Fri, 2009-03-13 at 14:11 +0100, Mathias Hasselmann wrote: I think you don't understand the problem. That might very well be the case. I had a look at the presentation that Alex linked to in the initial post in this thread. But I would have preferred a document that doesn't look at the issue from a database developer point of view. Here is a comment from the save file point of view. https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54 It seems wrong to work around broken file-systems on the application level. That only takes away pressure from the file-system developers to address the problem properly. I don't disagree, but on the other hand. Users are losing data as we speak. (See above ubuntu bug report) One compromise we could make it to only fsync in the case we're actually overwriting an existing file. This would mean that we don't risk loosing both the old and the new version of the file, you only lose new files. This case is far less common so the performance aspects are not as bad, and its also get rids of the worst failure mode. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Sven Neumann wrote: It seems wrong to work around broken file-systems on the application level. That only takes away pressure from the file-system developers to address the problem properly. How is the file system broken? Read the man page for write(). If you want to guarantee that file data will hit disk (or at least the disk's HW buffer) by a certain time, you need to call fsync() (or fdatasync(), where available). This isn't a Linux idiosyncrasy, even. POSIX specifies this. The only thing that's actually broken IIRC is ext3, in that a fsync() effectively acts as a full-FS sync() (see the Firefox 3.0/sqlite fiasco[1]), which is ridiculous. If anything should be fixed, *that* should be... as well as naive applications that think that open() - write() - close() is sufficient to get data to disk in a known amount of time. (Of course, ext3 won't ever be fixed, so... I guess we wait for ext4 use to become more widespread, and for btrfs to go stable.) -brian [1] http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/ ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On Fri, 2009-03-13 at 11:11 -0700, Brian J. Tarricone wrote: Sven Neumann wrote: It seems wrong to work around broken file-systems on the application level. That only takes away pressure from the file-system developers to address the problem properly. How is the file system broken? Read the man page for write(). If you want to guarantee that file data will hit disk (or at least the disk's HW buffer) by a certain time, you need to call fsync() (or fdatasync(), where available). The fact that its documented doesn't make it not broken. If you read the posix specs you'll see that its per specification for the implementation of fsync() to be empty. Now, we don't actually really need the data to be on the disk at a certain time. On the contrary, its really fine if its delayed. But, what we want is either the old file in place, or the new file in place, not the old file deleted, the metadata for the new file and the new file being empty. Thats what is broken, even if its allowed by POSIX. This isn't a Linux idiosyncrasy, even. POSIX specifies this. The only thing that's actually broken IIRC is ext3, in that a fsync() effectively acts as a full-FS sync() (see the Firefox 3.0/sqlite fiasco[1]), which is ridiculous. If anything should be fixed, *that* should be... as well as naive applications that think that open() - write() - close() is sufficient to get data to disk in a known amount of time. Broken is a wider concept than you think. Things that are fully up to some well documented spec can also be broken from the point of view of common sense. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On Fri, 2009-03-13 at 18:45 +0100, Alexander Larsson wrote:
One compromise we could make it to only fsync in the case we're actually
overwriting an existing file. This would mean that we don't risk loosing
both the old and the new version of the file, you only lose new files.
This case is far less common so the performance aspects are not as bad,
and its also get rids of the worst failure mode.
Attached is a patch that does this. It adds a
G_FILE_CREATE_SYNC_ON_CLOSE flag, and turns it on by default when
handling an overwrite. The same thing happens in g_file_set_contents().
I think we need to do this, at the minimum, because even if there are
some ext4 patches to fix up this issue they can be disabled, and from
what I understand XFS and btrfs have similar issues.
The question is, do we want this or the full patch?
Index: gio/glocalfileoutputstream.c
===
--- gio/glocalfileoutputstream.c (revision 7974)
+++ gio/glocalfileoutputstream.c (working copy)
@@ -69,6 +69,7 @@ struct _GLocalFileOutputStreamPrivate {
char *original_filename;
char *backup_filename;
char *etag;
+ gboolean sync_on_close;
int fd;
};
@@ -81,7 +82,7 @@ static gboolean g_local_file_output_st
GCancellable *cancellable,
GError**error);
static GFileInfo *g_local_file_output_stream_query_info (GFileOutputStream *stream,
- char *attributes,
+ const char *attributes,
GCancellable *cancellable,
GError**error);
static char * g_local_file_output_stream_get_etag (GFileOutputStream *stream);
@@ -190,6 +191,22 @@ g_local_file_output_stream_close (GOutpu
file = G_LOCAL_FILE_OUTPUT_STREAM (stream);
+#ifdef HAVE_FSYNC
+ if (file-priv-sync_on_close)
+{
+ if (fsync (file-priv-fd) != 0)
+ {
+ int errsv = errno;
+
+ g_set_error (error, G_IO_ERROR,
+ g_io_error_from_errno (errno),
+ _(Error writing to file: %s),
+ g_strerror (errsv));
+ goto err_out;
+ }
+}
+#endif
+
#ifdef G_OS_WIN32
/* Must close before renaming on Windows, so just do the close first
@@ -459,7 +476,7 @@ g_local_file_output_stream_truncate (GFi
static GFileInfo *
g_local_file_output_stream_query_info (GFileOutputStream *stream,
- char *attributes,
+ const char *attributes,
GCancellable *cancellable,
GError**error)
{
@@ -517,6 +534,8 @@ _g_local_file_output_stream_create (con
stream = g_object_new (G_TYPE_LOCAL_FILE_OUTPUT_STREAM, NULL);
stream-priv-fd = fd;
+ if (flags G_FILE_CREATE_SYNC_ON_CLOSE)
+stream-priv-sync_on_close = TRUE;
return G_FILE_OUTPUT_STREAM (stream);
}
@@ -562,6 +581,8 @@ _g_local_file_output_stream_append (con
stream = g_object_new (G_TYPE_LOCAL_FILE_OUTPUT_STREAM, NULL);
stream-priv-fd = fd;
+ if (flags G_FILE_CREATE_SYNC_ON_CLOSE)
+stream-priv-sync_on_close = TRUE;
return G_FILE_OUTPUT_STREAM (stream);
}
@@ -992,6 +1013,7 @@ _g_local_file_output_stream_replace (con
if (fd == -1 errno == EEXIST)
{
+ flags |= G_FILE_CREATE_SYNC_ON_CLOSE;
/* The file already exists */
fd = handle_overwrite_open (filename, etag, create_backup, temp_file,
flags, cancellable, error);
@@ -1022,6 +1044,8 @@ _g_local_file_output_stream_replace (con
stream = g_object_new (G_TYPE_LOCAL_FILE_OUTPUT_STREAM, NULL);
stream-priv-fd = fd;
+ if (flags G_FILE_CREATE_SYNC_ON_CLOSE)
+stream-priv-sync_on_close = TRUE;
stream-priv-tmp_filename = temp_file;
if (create_backup)
stream-priv-backup_filename = create_backup_filename (filename);
Index: gio/gioenums.h
===
--- gio/gioenums.h (revision 7974)
+++ gio/gioenums.h (working copy)
@@ -164,13 +164,17 @@ typedef enum {
*You can think of it as unlink destination before
*writing to it, although the implementation may not
*be exactly like that. Since 2.20
+ * @G_FILE_CREATE_SYNC_ON_CLOSE: If possible, try to ensure
+ *that all data is on disk before returning. For local
+ *files this means calling fsync() before close.
*
* Flags used when an operation may create a file.
*/
typedef enum {
G_FILE_CREATE_NONE= 0,
G_FILE_CREATE_PRIVATE = (1 0),
- G_FILE_CREATE_REPLACE_DESTINATION = (1 1)
+ G_FILE_CREATE_REPLACE_DESTINATION = (1 1),
+ G_FILE_CREATE_SYNC_ON_CLOSE = (1 2)
} GFileCreateFlags;
Index: configure.in
===
--- configure.in (revision 7974)
+++ configure.in (working copy)
@@ -563,6 +563,7 @@ AC_CHECK_FUNCS(mmap)
AC_CHECK_FUNCS(posix_memalign)
AC_CHECK_FUNCS(memalign)
AC_CHECK_FUNCS(valloc)
+AC_CHECK_FUNCS(fsync)
AC_CHECK_FUNCS(atexit on_exit)
Index: glib/gfileutils.c
Re: fsync in glib/gio
On Fri, 2009-03-13 at 09:15 -0400, Morten Welinder wrote: F*** POSIX allows this! A program that does open-write-close- rename should not be left with an empty file in case something goes wrong. The old file, or the new file. Anything else is insane and by extension the kernel developers and their ancestors. 100% agreed. Has anyone actually debugged why this happens? The kernel must surely ensure that even if it reorders data/metadata requests, it will do so in sensible ways only, doesn't it? Federico ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On 03/13/2009 05:16 PM, Alexander Larsson wrote: On Fri, 2009-03-13 at 15:05 -0600, Federico Mena Quintero wrote: On Fri, 2009-03-13 at 09:15 -0400, Morten Welinder wrote: F*** POSIX allows this! A program that does open-write-close- rename should not be left with an empty file in case something goes wrong. The old file, or the new file. Anything else is insane and by extension the kernel developers and their ancestors. 100% agreed. Has anyone actually debugged why this happens? The kernel must surely ensure that even if it reorders data/metadata requests, it will do so in sensible ways only, doesn't it? Its well explained in the various discussions about this. Essentially, the metadata for the rename is written to disk, but the data in the file is not (yet, due to delayed allocation) and then the system crashes. On fsck we discover the file is broken (no data) and set the file size to 0. That's clearly a broken filesystem (screw standards. If it doesn't do what users expect, it's broken). Why work around it in gio? Have the filesystem guys fix it for whatever that means. All we need the few major distros handling it properly. behdad ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Alexander Larsson wrote: On Fri, 2009-03-13 at 11:11 -0700, Brian J. Tarricone wrote: Sven Neumann wrote: It seems wrong to work around broken file-systems on the application level. That only takes away pressure from the file-system developers to address the problem properly. How is the file system broken? Read the man page for write(). If you want to guarantee that file data will hit disk (or at least the disk's HW buffer) by a certain time, you need to call fsync() (or fdatasync(), where available). The fact that its documented doesn't make it not broken. If you read the posix specs you'll see that its per specification for the implementation of fsync() to be empty. That's not the point I'm trying to make. It may be 'stupid' behavior, but it's at least specified. Saying the filesystem guys should fix their filesystems to be less lame just doesn't work, as they're compliant with the spec. So either the app developer can write their save routines to be robust *in the face of the spec*, or they can 'hope' that every new FS adopts a restriction on behavior that isn't specified anywhere, and every old FS is modified and updated to follow this fantasy restriction. Doesn't that sound a bit like unreasonable wishful thinking? Now, we don't actually really need the data to be on the disk at a certain time. On the contrary, its really fine if its delayed. But, what we want is either the old file in place, or the new file in place, not the old file deleted, the metadata for the new file and the new file being empty. Thats what is broken, even if its allowed by POSIX. Sure, but that's just a special case. So you (as the app developer) recognise this, understand how the spec interacts with your use-case, and write robust code accordingly. Or, you take the the spec/kernel/FS is broken approach, and try to get a guarantee specified for the special case, something like in the case where a file is renamed over top of an existing file, the source file must be flushed to disk before the rename takes place. And then the app developer doesn't have to worry about it, because the implementation should do the right thing. Your patch to gio takes the first approach, which is fine, I think, if unfortunate in the sense that it forces behavior that may not be desired. A user of g_file_set_contents() may be writing a temp file or something that they don't care all that much about, and doing so arguably reduces performance. Of course, g_file_set_contents() is a decently high-level abstraction, so one could argue that people who want finer control over how the file gets written should use gio or open/write/close directly. This isn't a Linux idiosyncrasy, even. POSIX specifies this. The only thing that's actually broken IIRC is ext3, in that a fsync() effectively acts as a full-FS sync() (see the Firefox 3.0/sqlite fiasco[1]), which is ridiculous. If anything should be fixed, *that* should be... as well as naive applications that think that open() - write() - close() is sufficient to get data to disk in a known amount of time. Broken is a wider concept than you think. Things that are fully up to some well documented spec can also be broken from the point of view of common sense. Yeah, I'd totally agree. But in the absence of an ability to change the spec, it's best to try to make things work as well as they can within the spec, no? It seems like some people are advocating well, today everyone uses ext3, and there's no problem, so we shouldn't do this because it'll reduce performance there. And of course, a year from now (or less! obviously some already are), I'm sure most desktop distros will be shipping with ext4 default. (And I could be wrong, but it seems to me that ext3 is the only FS that, by coincidence will usually be immune to this problem, and, also coincidentally, is one of the only FSes that has crappy fsync() performance.) I dunno... my vote/opinion would be to have a _SYNC flag, leave async as the default, and force _SYNC for g_file_set_contents() (maybe?) and for cases in gio where we know a rename is going to overwrite an existing file (if it's possible to know that without a perf hit). -brian ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Behdad Esfahbod wrote: Its well explained in the various discussions about this. Essentially, the metadata for the rename is written to disk, but the data in the file is not (yet, due to delayed allocation) and then the system crashes. On fsck we discover the file is broken (no data) and set the file size to 0. That's clearly a broken filesystem (screw standards. If it doesn't do what users expect, it's broken). Why work around it in gio? Have the filesystem guys fix it for whatever that means. All we need the few major distros handling it properly. There are different definitions of broken. I might consider it broken if my glib/gio application which writes out thousands of little files and suddenly starts taking twice as long as my Perl program. fsync() has a real measurable cost. The documentation for the file systems is usually quite clear, although people may not understand it. When a file system such as ext3 offers different journal modes, it's presumed that the user understands the effect of their choice. It the user must be absolutely safe - they should use 'journal' mode - but this may be slow, as all data is written to disk at least twice. If 'ordered' mode is used, this means they are willing to accept lesser guarantees for increased performance. The ext3 'ordered' mode works pretty well - data before metadata, and metadata updates are ordered. But - it's not perfect! What order is the data written in? Do you intend to patch glib/gio if somebody reports that glib/gio used write() to one part of a file, then another part of a file, then pulls the plug, and is able to prove to you that it is possible for the second write to finish while the first write hasn't started? Will you call it absolutely broken and demand a file system fix? The ext3 'writeback' mode provides even fewer guarantees. Meta data is ordered, data is not. The behaviour you are talking about right now seems to be 'writeback' mode (not sure - I guess ext4 is doing 'writeback' mode by default otherwise I don't understand the complaint?). Tell your users if they expect the right thing to use 'journal' mode. glib/gio cannot and should not be second guessing the file system choice of the user. Taking this argument to its extreme, you may as well run fsync() after every single I/O operations that performs a modification. This would be horrible for performance, and the user has this capability already by defining the file system as completely journalled and using synchronous writes. They don't need glib/gio to simulate this. My opinion is that glib/gio shouldn't be doing this stuff. The problem is not with glib/gio. glib/gio should offer an fsync() wrapper (not sure if it does or not - I don't use it), such that applications with special requirements such as a database application, can use fsync() at strategic points where the application wishes to make greater promises than the file system. A database file applies here. For databases specifically, fsync() on close() is not good enough. fsync() needs to be done at any point that the data needs to be consistent and written to disk before the application continues to do another write(). glib/gio cannot guess where these points are. Putting fsync() on close() is a hack. Just my opinion. :-) Cheers, mark -- Mark Mielke m...@mielke.cc ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
Mark Mielke wrote: Putting fsync() on close() is a hack. Hmm - Looking at the patch, I don't see it doing fsync() on close() - I should have read from the beginning instead of reacting to the one person calling the file system semantics broken. :-) Definitely - any file system operations that *requires* file system integrity, should make use of fsync(). The write new file and rename into place is a good candidate, and fsync() should definitely be done here. Wishful thinking about some theoretical system which is both efficient, deployed on all systems everywhere, and does fully synchronous writes for both data and metadata is pointless. :-) Note that this doesn't matter what file system is used. Older non-journalled file systems had the same problem. The only file systems that are completely safe are the ones that do full journalling of all data. (Those also tend to be the slowest file systems or modes although some file systems may be breaking this trend...) Just please don't *always* fsync(). Only where it matters. As per the patch, I think it tries to do it where it matters and looks fine? Cheers, mark -- Mark Mielke m...@mielke.cc ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On Fri, 2009-03-13 at 22:16 +0100, Alexander Larsson wrote: Its well explained in the various discussions about this. Essentially, the metadata for the rename is written to disk, but the data in the file is not (yet, due to delayed allocation) and then the system crashes. On fsck we discover the file is broken (no data) and set the file size to 0. This reminds me a lot of http://bugzilla.gnome.org/show_bug.cgi?id=562396 - a problem with Nautilus metadata. You start a copy operation, but if you do a read before the copy is done, then you get the old data. You should wait for the copy to be done first, but anyway, my point is... My point is that the kernel could perfectly well ensure that metadata operations that depend on data operations will not be reordered. Don't rename a file in a directory if we have outstanding writes for the inode, or something. (After the rename, do you need to open the directory and fsync it? You can't open directories for writing...) Federico ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: fsync in glib/gio
On Fri, Mar 13, 2009 at 06:20:57PM -0600, Federico Mena Quintero wrote: My point is that the kernel could perfectly well ensure that metadata operations that depend on data operations will not be reordered. I think that's how ext3 works by default (data=ordered mount option). http://en.wikipedia.org/wiki/Ext3#Journaling_levels Don't know if the same behaviour is available in ext4 Berto ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
