Re: GLib plans for next cycle
On 09/06/11 10:05, Alexander Larsson wrote: On Thu, 2011-09-01 at 15:16 -0600, Ryan Lortie wrote: Another option is to use library load constructors to run the initialisation we need to do. That's certainly possible on Windows systems and anything using GCC. I'm not sure if it's possible to do it in a portable way from pure C, though. In my research for how to implement a resource system for glib I've been looking at this. I think its should be possible to do this for all the main platforms we're targeting (linux, elf unixes, win32, osx). There might be some ancient proprietary unix where it will not work (but probably not as this is a C++ requirement too). So, while the code would be kinda hacky to make this work with all weird setups required (like pragmas, ick) it should be quite possible to rely on. In HarfBuzz I'm using the C++ compiler without linking to libstdc++. I found it as very rewarding experience. You get library constructor/destructors for free there. I think this approach to library design has serious merit worth considering. b I have some notes somewhere on how it can be done for all platforms. Will write it up some day. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Tue, 2011-09-06 at 10:14 -0400, Behdad Esfahbod wrote: In HarfBuzz I'm using the C++ compiler without linking to libstdc++. I found it as very rewarding experience. You get library constructor/destructors for free there. I think this approach to library design has serious merit worth considering. Ha! I was just writing an email along the lines of why don't we just compile a tiny bit of C++ code? before thinking oh... but then we'd have to link libstdc++ and closing the compose window. Good to hear :) Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Tue, 2011-09-06 at 10:48 -0400, Behdad Esfahbod wrote: On 09/06/11 10:42, Ryan Lortie wrote: On Tue, 2011-09-06 at 10:14 -0400, Behdad Esfahbod wrote: In HarfBuzz I'm using the C++ compiler without linking to libstdc++. I found it as very rewarding experience. You get library constructor/destructors for free there. I think this approach to library design has serious merit worth considering. Ha! I was just writing an email along the lines of why don't we just compile a tiny bit of C++ code? before thinking oh... but then we'd have to link libstdc++ and closing the compose window. Yeah, the trick is to pass -fno-rtti -fno-exceptions to g++. In fact, libtool already has code to do this if your C compiler turns out to be a C++ compiler. It's the macro _LT_COMPILER_NO_RTTI. I've emailed libtool maintainers to ask to make this public. This is specific to gnu g++ though. If you want to target only that then you might as well just use the gnu c extensions to do library constructors in plain C. I have plain C ways to implement this for most systems, although we could use the C++ trick as a fallback. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Wed, 2011-08-31 at 12:17 -0700, David Zeuthen wrote: Hi, On Wed, Aug 31, 2011 at 1:06 PM, Ryan Lortie de...@desrt.ca wrote: - glib_get_worker_context() s/glib/g/ ? glib_* sounds like it's for glib-internal-only use, which I don't see any reason for. It's definitely useful outside of glib. My original intention is that it *would* be glib-private (like glib_gettext() for example). It's true that other libraries could find it useful, though -- dconf would certainly put itself on this list. I'm just slightly worried about this thread becoming contaminated (think: mainloop blocking) through use by poorly written libraries. That's my concern too. In fact, for reasons like this, I'm not happy about on having GDBus use it unless someone can demonstrate significant savings - and, honestly, in these days where the CPUs worth targeting with GLib are all multi-core, putting all of our crap in a single worker thread is probably a performance hit (but I doubt it's measurable). Maybe the time to discuss pulling in an api something like Grand Central Dispatch has arrived? :) Kevin David ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On 08/31/2011 03:31 PM, Kevin Fox wrote: Maybe the time to discuss pulling in an api something like Grand Central Dispatch has arrived? :) I was thinking it would be nice to integrate with GThreadPool somehow, but I didn't have any specific idea of how that would work... -- Dan ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
An update. On Wed, 2011-08-31 at 11:50 -0400, Ryan Lortie wrote: I'm working on some plans for things I want to do in GLib at the start of the next cycle. I'd do them now, but it's getting late. I created a wip/glib-next branch and started working on a lot of these ideas. I started by merging a couple of awesome patches from Dan that consisted of making thread support mandatory and deleting a whole tonne of totally pointless threads-not-initialised code from gio. Here's the status of items from my list: - merge libgthread into libglib Seemed like a popular idea, so done. - consider removing support for user-specified thread implementations I did this, but I possibly regret it. See below for more discussion. - merge libgmodule into libglib Not done due to push-back and general pointlessness. - glib_get_worker_context() Done. - remove the racy non-threaded case of GMainContext Done. - make GMainContext use the GLib worker context Done, and I think it's okay, but it may contain a race. It's difficult to test, though because of a glibc bug that I think I've run into. - make GDBus use the GLib worker context - make GSettings use the GLib worker context - make GTimeZone use the worker context Pending. - land some very minor (and 100% compatible) changes to the GSource API see https://bugzilla.gnome.org/show_bug.cgi?id=657729 The proposed API changes are on the branch. I've ported the timeout source (which was a net reduction in code). - with the updated GSource API we can start looking at epoll in a meaningful way Big work to be done here. This will take a while. :) = On the topic of removing support for custom thread implementations: this simplifies things a fair bit, and would let us get rid of a lot of ugly code in gthread.h. In some cases it could theoretically be a performance regression to do this, though. Here's why: The POSIX implementation of GMutex uses the native pthreads mutex quite directly -- so much so that it uses the pthread functions directly as the entries for the thread implementation vtable. That vtable is exposed as a public global variable and macros in gthread.h do this: #define g_mutex_lock(mutex) \ if (g_thread_supported ())\ vtable.lock (mutex);\ else \ (do nothing); Because the vtable.lock function is a direct pointer to the pthreads locking function, this has the potential to be faster than calling a real (normal) g_mutex_lock() function that then calls pthreads. It could be that 2 normal function calls are actually faster than looking up the address of the vtable and doing a computed jump on a pointer contained therein -- I haven't checked yet, but I doubt the difference is substantial either way. The if (g_thread_supported()) thing is a performance killer, but I think it's not problematic for a couple of reasons. It basically comes down to the fact that this will hopefully be (statically) TRUE in the future, and in some cases already is. There are a couple of approaches we could take here (keeping in mind that the global variable will always have to be exposed for purposes of ABI stability, even if we change the API): - keep things how they are now. This has the side effect of still allowing the thread implementation to be replaced (which is questionably useful, of course). - remove the hackery and use normal functions, accepting the (extremely small) potential performance hit - use defines that call the pthread functions directly when we're on POSIX, just like we do for the atomic ops. This forever binds us to never doing anything more interesting, since we will have binaries out there with pthread calls directly embedded. One thing worth noting is that there is at least one custom thread implementation in existence: the errorchecking mutex implementation. It could be nice to keep that around. Another question about threads is if we should still require g_thread_init() to be called. There is currently a lot of setup code that needs to be run and I'm not sure if it's possible to get rid of it all. Right now, how it works on the branch is that something that needs the thread system to be initialised will call g_thread_init_glib() directly, which will do the work. It might be nice to get rid of the init call entirely and just have threads that are always working, right from the start. That would certainly be necessary in order to drop the g_thread_supported() conditional. That would require a bit more work. Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Thu, Sep 1, 2011 at 3:17 PM, Ryan Lortie de...@desrt.ca wrote: - glib_get_worker_context() Done. Would strongly prefer this API is not public. See: https://bugzilla.gnome.org/show_bug.cgi?id=657992 ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Thu, Sep 1, 2011 at 3:17 PM, Ryan Lortie de...@desrt.ca wrote: An update. On Wed, 2011-08-31 at 11:50 -0400, Ryan Lortie wrote: I'm working on some plans for things I want to do in GLib at the start of the next cycle. I'd do them now, but it's getting late. I created a wip/glib-next branch and started working on a lot of these ideas. There's a huge amount of stuff here - how much of this is really dependent? Feature branches are just way easier to review than one big stuff. The proposed API changes are on the branch. I've ported the timeout source (which was a net reduction in code). In particular I'd *really* like to see this as a separate branch. - with the updated GSource API we can start looking at epoll in a meaningful way Big work to be done here. This will take a while. :) Right...defer this? - use defines that call the pthread functions directly when we're on POSIX, just like we do for the atomic ops. This forever binds us to never doing anything more interesting, since we will have binaries out there with pthread calls directly embedded. Binaries - for a specific platform, yes. Since we know Linux/glibc works - and these are in fact FOSS projects that we can contribute to, I don't see a problem just emitting pthread calls. One thing worth noting is that there is at least one custom thread implementation in existence: the errorchecking mutex implementation. It could be nice to keep that around. I think glib should move more towards having separate normal and full on super slow debug stuff modes, similar to how the lockdep stuff is a config option for Linux. Another question about threads is if we should still require g_thread_init() to be called. There is currently a lot of setup code that needs to be run and I'm not sure if it's possible to get rid of it all. Right now, how it works on the branch is that something that needs the thread system to be initialised will call g_thread_init_glib() directly, which will do the work. If you're doing it in g_main_context_default(), that seems almost equivalent to not requiring it =) ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
hi Colin, On Thu, 2011-09-01 at 16:13 -0400, Colin Walters wrote: There's a huge amount of stuff here - how much of this is really dependent? Feature branches are just way easier to review than one big stuff. All of it except the source API changes is dependent on the libgthread merge. My intention here is to mass-land this stuff after we branch the stable GLib and use this to start the next cycle, thus the name glib-next. I do expect there will be significant rebasing as things are reviewed. The proposed API changes are on the branch. I've ported the timeout source (which was a net reduction in code). In particular I'd *really* like to see this as a separate branch. This was a separate branch, actually, but I just nuked it to reduce the noise of all the branches I've been pushing. I'll split it back out and push it again. - with the updated GSource API we can start looking at epoll in a meaningful way Big work to be done here. This will take a while. :) Right...defer this? I consider GLib to be fully baked for this cycle -- strictly bugfixes only at this point. Everything in this email and on the branch is for the start of next cycle. - use defines that call the pthread functions directly when we're on POSIX, just like we do for the atomic ops. This forever binds us to never doing anything more interesting, since we will have binaries out there with pthread calls directly embedded. Binaries - for a specific platform, yes. Since we know Linux/glibc works - and these are in fact FOSS projects that we can contribute to, I don't see a problem just emitting pthread calls. It might become a problem later if we decide that we want to run a program with debugging mutexes or something. Consider the case, particularly, where we do things like passing a GCond* or GMutex* across a library barrier -- if one library was compiled hard against pthreads and the other is using some other library (like debugging mutexes) then we're in a substantial amount of trouble. We'd essentially be committing to we will always use pthreads mutexes as GMutex* and never anything else. Your point that glibc is a free software project is a good one though -- why not just add the debug support there? I think glib should move more towards having separate normal and full on super slow debug stuff modes, similar to how the lockdep stuff is a config option for Linux. I actually think the opposite. I'd prefer if we could enable super-slow debug mode with an environment variable. It would make it way easier to get debugging information out of users who are experiencing a problem and don't want to rebuild their system glib or their application. That's a story for another day... Another question about threads is if we should still require g_thread_init() to be called. If you're doing it in g_main_context_default(), that seems almost equivalent to not requiring it =) Almost being the key word, of course. If people expect that the threading system is always implicitly initialised and then some day come across the weird case where it isn't, they'll be quite confused/annoyed. Ideally I'd like to solve this problem by always having the thread system ready for use without explicit initialisation. At the end of the day, all initialisation that we do can be done by a GOnce at the first time that we go to use the thing that needs to be initialised. Even on Windows, where we may not necessarily be able to have static mutexes without initialisation, we can one-time bootstrap ourselves using a spinlock based on atomic integer operations provided by the native Windows interlocked API -- and I doubt it would even come to that. Another option is to use library load constructors to run the initialisation we need to do. That's certainly possible on Windows systems and anything using GCC. I'm not sure if it's possible to do it in a portable way from pure C, though. Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Thu, Sep 1, 2011 at 5:16 PM, Ryan Lortie de...@desrt.ca wrote: All of it except the source API changes is dependent on the libgthread merge. I see that now - makes sense. This was a separate branch, actually, but I just nuked it to reduce the noise of all the branches I've been pushing. I'll split it back out and push it again. Great, thanks! I should also mention I looked briefly through the source stuff and it looked really nice - you have good ideas here and the patches are pretty clean. Everything in this email and on the branch is for the start of next cycle. Yeah, I more meant that ideally the source stuff lands after. I actually think the opposite. I'd prefer if we could enable super-slow debug mode with an environment variable. It would make it way easier to get debugging information out of users who are experiencing a problem and don't want to rebuild their system glib or their application. If the cost is small, I agree environment variables are fine. But the cost is not always small even to just have the code around (think about reference count tracking). Another option is to use library load constructors to run the initialisation we need to do. That's certainly possible on Windows systems and anything using GCC. I'm not sure if it's possible to do it in a portable way from pure C, though. This might be the way to go if we can make it cheap. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
GLib plans for next cycle
I'm working on some plans for things I want to do in GLib at the start of the next cycle. I'd do them now, but it's getting late. I've tried some aborted attempts at a few of these things in the past, but they were all made complicated and ugly by one big factor: we don't have always-on support for creating threads in libglib. So here's my list -- both as a solicitation of feedback and as a way to make sure I don't forget it. - merge libgthread into libglib The amount of code in libgthread is quite small and almost everything on earth is linking against it anyway. Not having threading support built into libglib is preventing us from doing some nice things that we could do otherwise. - consider removing support for user-specified thread implementations - merge libgmodule into libglib This isn't really important, but why not? - glib_get_worker_context() Add a new function for use within glib (and gobject, gio): GMainContext * glib_get_worker_context (void); This function would create a thread (which we can do if libgthread is in libglib) and setup a GMainLoop running in it, returning the context. Future calls would return the same context. This gives us somewhere to take care of things in the background and gives us the benefit of always having a mainloop running (so we no longer have to go out of our way to support the non-mainloop use case of GLib). - remove the racy non-threaded case of GMainContext We currently have a long-standing race condition in GMainContext that occurs when dealing with signals in the single-threaded case only. This is what used to make gtester randomly hang sometimes (before a workaround was added to gtester). Since we have libgthread always-on, we can solve that problem once and for all, in addition to ripping out a bunch of rarely-used code. - make GMainContext use the GLib worker context GMainContext currently uses its own internal worker thread to deal with signals in the threaded case (which is why they're reliable). It could share the worker thread. - make GDBus use the GLib worker context GDBus uses a worker thread. It could also share. - make GSettings use the GLib worker context Here's the 3rd separate worker thread in typical GNOME programs. Merge it as well. - make GTimeZone use the worker context This would allow us to detect changes in /etc/localtime via inotify instead of polling it each time we create a new GDateTime. - land some very minor (and 100% compatible) changes to the GSource API see https://bugzilla.gnome.org/show_bug.cgi?id=657729 - with the updated GSource API we can start looking at epoll in a meaningful way Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On 08/31/2011 11:50 AM, Ryan Lortie wrote: - merge libgthread into libglib Getting rid of libgthread or moving any of its symbols into libglib is an ABI break on some platforms. Of course, since there are only 2 exported symbols (g_thread_init and g_thread_init_with_errorcheck_mutexes), I guess you can just move everything else in libgthread into libglib, and keep libgthread with those methods as no-ops. https://bugzilla.gnome.org/show_bug.cgi?id=616754 has configure patches to make building with thread support mandatory, and gobject and gio patches to remove the !g_thread_supported() codepaths. - merge libgmodule into libglib This isn't really important, but why not? Well, the same ABI issues again. If you don't have a use case it might be better to not bother. (Or, you could rename it in glib. Or have it as a private API in glib, with libgmodule just having public API wrapping it.) - glib_get_worker_context() s/glib/g/ ? glib_* sounds like it's for glib-internal-only use, which I don't see any reason for. It's definitely useful outside of glib. -- Dan ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Wed, 2011-08-31 at 12:52 -0400, Dan Winship wrote: On 08/31/2011 11:50 AM, Ryan Lortie wrote: - merge libgthread into libglib Getting rid of libgthread or moving any of its symbols into libglib is an ABI break on some platforms. Of course, since there are only 2 exported symbols (g_thread_init and g_thread_init_with_errorcheck_mutexes), I guess you can just move everything else in libgthread into libglib, and keep libgthread with those methods as no-ops. My exact plan is to leave the library installed with the two functions calling equivalent functions in libglib (or doing nothing at all if we decide to scrap user-provided thread implementations as well). Next step is to modify the libgthread.pc file so that newly-compiled programs don't even bother linking the library anymore. We could also use the symlinking trick -- make libgthread a symlink to libglib, thus maintaining compatibility. There was a lot of talk about how this is a valid technique when we were considering the libgobject/libgio merges. - merge libgmodule into libglib This isn't really important, but why not? Well, the same ABI issues again. If you don't have a use case it might be better to not bother. (Or, you could rename it in glib. Or have it as a private API in glib, with libgmodule just having public API wrapping it.) Since I don't have a compelling case for it, it might indeed be best to just leave well enough alone. - glib_get_worker_context() s/glib/g/ ? glib_* sounds like it's for glib-internal-only use, which I don't see any reason for. It's definitely useful outside of glib. My original intention is that it *would* be glib-private (like glib_gettext() for example). It's true that other libraries could find it useful, though -- dconf would certainly put itself on this list. I'm just slightly worried about this thread becoming contaminated (think: mainloop blocking) through use by poorly written libraries. Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
Hi, On Wed, Aug 31, 2011 at 1:06 PM, Ryan Lortie de...@desrt.ca wrote: - glib_get_worker_context() s/glib/g/ ? glib_* sounds like it's for glib-internal-only use, which I don't see any reason for. It's definitely useful outside of glib. My original intention is that it *would* be glib-private (like glib_gettext() for example). It's true that other libraries could find it useful, though -- dconf would certainly put itself on this list. I'm just slightly worried about this thread becoming contaminated (think: mainloop blocking) through use by poorly written libraries. That's my concern too. In fact, for reasons like this, I'm not happy about on having GDBus use it unless someone can demonstrate significant savings - and, honestly, in these days where the CPUs worth targeting with GLib are all multi-core, putting all of our crap in a single worker thread is probably a performance hit (but I doubt it's measurable). David ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: GLib plans for next cycle
On Wed, 2011-08-31 at 19:58 +0200, Jannis Pohlmann wrote: TBH, I don't think the name alone will prevent misuse; in particular so as the glib prefix is no obvious internal-only hint (e.g. glib_check_version() is not private either). Wouldn't a note in the API docs suffice? We have a few functions like glib_gettext() that are glib internal API (enforced by not being defined in an installed header). I envision this falling into the same category. Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
