Ok, I'll try to answer this :-) On Nov 3, 2009, at 11:20, Simon Marlow wrote:
> On 02/11/2009 21:11, Axel Simon wrote: > >> The story is as follows: >> >> Calling g_object_unref, the principal finalizer we attach to any >> ForeignPtr, will decrement the reference count and immediately >> finalize >> an object if this counter is zero. This can pose problems in three >> ways: >> >> a) Storing any callback or other memory region in the GObject will >> install a Haskell finalizer (freeStablePtr or free function pointer) >> that is called when the GObject is finalized. After our last long >> conversation about the new concurrent GC, we changed Gtk2Hs so >> that it >> passes the addresses of the C functions of the ghc runtime as >> finalizers, so Haskell is not called and all is well. (I hope I >> recall >> this correctly.) > > Correct, as far as I recall. You have to ensure that Haskell can't > be called from a C finalizer, or else use Haskell finalizers from > Foreign.Concurrent. Which is why I was > surprised that my backtrace showed that Haskell was being called > from a C finalizer! I just realized that we haven't changed everything to use direct calls to C finalizers but to Haskell finalizers. I assume that this is because Haskell still got called from within the GC. If it is still called from within the GC, then some signals are being emitted as a result of finalizing an object. I probably need an example where this goes wrong. I don't know if it is possible to run the Haskell program in gdb and get a traceback of the C functions that called Haskell during GC. However, it might just be a very plain bug in that we bound a function as 'unsafe' when, in fact, it may trigger a callback to Haskell. > >> b) It is possible to add a Haskell finalizer function to a GObject. >> Since these Haskell finalizers are called from the GC, they have to >> re-enter the Haskell RTS which will bomb. Solution: remove the >> possibility to add such a finalizer function. > > So is this currently done or not? Why do I see a Haskell function > being called by a finalizer? Well, as I said, probably an 'unsafe' function that should be declared 'safe'. We still provide the possibility, though, to attach a function that is called when an object dies. I don't think many people use this, but I don't know if we can categorically avoid this situations: some GObject in Gtk+ may trigger some 'disconnect some child object' if a finalizer is run. I don't know of any specific cases, though. >> c) Xlib is not thread-safe but has no thread-local storage so it >> can be >> called from several threads if they don't access Xlib at the same >> time. >> Win32 API is not thread-safe *and* uses thread-local storage so >> that it >> is impossible to call it from more than one thread. >> >> Since a GObject may contain X or Win32 ressources, these might be >> freed >> from a thread calling g_object_unref. Thus, if the ghc GC calls >> g_object_unref from several threads, it is inevitable that Xlib or >> Win32 >> is called from several threads. > > But the GC never invokes finalizers from several threads! It is > true that it doesn't give you any guarantee about *which* thread it > will run finalizers on, but the finalizers themselves run > sequentially. Oh. Well, it wouldn't make things much worse if finalizers were run concurrently. I would have to wrap g_object_unref in a function that acquires a global lock, but that's all. Why is this not done? > However, what may happen is that the GC runs at the same time as a > (safe) foreign call, which may well cause the problems you > describe. This is not new, but the fact that the GC now runs > finalizers *is* new. More on this below... > >> This is what you observe below when get >> an Xlib error. I have a patched version of Gtk2Hs lying around that >> replaces the finalizer 'g_object_unref' with a wrapper function that >> first acquires a global Gtk lock before calling g_object_unref. >> This may >> severely stall the GC but works well for Xlib. On Windows, this still >> doesn't work. Nor does it solve b). >> >> The only solution I can think of is to insert all GObjects that are >> finalized during GC into a list and to finialize all these objects by >> calling a function (say 'runFinalizers') from the main Gtk thread. >> This >> function can be called from Haskell in a 'safe' way so that >> callbacks to >> Haskell are no problem. >> >> The problem that remains is how to ensure that 'runFinalizers' is >> called >> after each GC. When using Gtk+, I could add the 'runFinalizer' >> function >> to the main loop of Gtk. Thus, when the GUI resumes after a GC, it >> will >> finalize all objects that were dead during the last GC. >> >> However, if we ever split off Cairo or Pango (two modules that may be >> used independently of the GUI stuff) then there is no more Gtk+ main >> loop. Thus, we would need another way to run the 'runFinalizers' C >> function. >> >> One idea I had to this end is a hook into the ghc runtime: We need to >> install a C pointer to a function that is called at the end of >> each GC. >> Additionally, we need to ensure that this function is called by one >> specific OS thread. Alternatively, we would need a Haskell-side >> hook to >> install a OS-thread bound Haskell function that, in turn, calls the C >> function. I talked to Duncan about this but he was skeptical that GHC >> headquarters would be happy about this. > > Calling a C function after GC would be quite easy to add. However, > ensuring it gets run by a specific OS thread would be much harder - > the OS thread you want to use might be in the middle of a foreign > call, for example. > >> <rant> >> In fact, the situation is a bit contradictory: As an application >> developer you are responsible to carefully call all your functions >> from >> an (OS-)bounded thread if your C library isn't thread-safe and >> then the >> GC comes along and calls concurrently with any number of OS >> threads into >> the C library to run the finalizers. This is not quite fair onto the >> programmer. >> <rant/> > > So while this is slightly incorrect (the GC doesn't run finalizers > concurrently), I agree that there is a problem in that you can't > effectively synchronise between C finalizers and other activity. > > I suggest that the C finalizers should be very simple: just add the > tasks to be done onto a queue protected by a mutex, and have the > main loop process the queue. The mutex is necessary not to protect > against multiple finalizers running concurrently, but to protect > against finalizers running concurrently with foreign calls (e.g. > the gtk main loop). > > This sounds like what you proposed above, and in order to make this > work the GC needs to inform the main loop when more finalizers have > been added. I'm not sure of the best way to do this. I'm not quite following: who is in charge of this list? The GC? As it stands, I can already implement what you suggest above: I replace each call of the GC to g_object_unref with a function that enqueues the finaliasation task into the Gtk+ queue. As I said, the problems with this is that I cannot guarantee that the application program will every run the Gtk main loop. I can only guarantee that in real GUI programs, not in those that use Gtk2Hs to render some graphics into a file. Since the problems only really occur when objects with handles to Xlib or Win32 are finalized this would be a solution and is implementable without any change to ghc. I'm just weary that there is some object whose finalizer I do not enqueue in the main Gtk loop and which for some reason calls either Xlib/Win32 or Haskell. > I wonder if for the time being you should just revert to using > Haskell finalizers. They are less efficient and prompt, but they > do support proper synchronisation. It seems I'm using Haskell finalizer. So you might just have observed an unsafe/safe declaration bug in Gtk2Hs. However, the Xlib warning is the real thing that cannot so easily be worked around. >> Another idea would be to have a new type of finalizers that takes a >> thread id and that the GC then guarantees to only run from the given >> thread id. This would push the burden into the GC implementation. >> >> Any thoughts appreciated. With the current solution, I don't think >> there's a way for us to make Gtk2Hs thread-safe except for the >> hack of >> using the Gtk+ main loop to run finalizers. > > Can I suggest that first of all there should be a warning on the > gtk2hs page and in the documentation about the use of -threaded; if > I'd known that it wasn't currently supported I would have avoided it. Yes, I agree, we're not very upfront about this... > Secondly, I'm happy to help here. I think we ought to have multi- > threaded GUI support, and it's worth putting in some effort to make > it work. Let me know how you'd like to proceed. Could you say why it is so hard to associate a finalizer with a thread id and ensure that the finalizer is run from that particular thread? It seems easier to me now than before, when I though the GC would execute finalizers concurrently rather than enqueuing them in a list. It would be ok to associate a ForeignPtr with a Haskell thread id (which in turn is bound to the right OS thread). You could keep a list of finalizers on a per-thread-id basis and add a process that executes these finalizers into the running queue of that thread. With this approach, it wouldn't matter that the associated OS thread is currently running in C land. I'm probably naive about the complexity of this approach, but what I would implement in Gtk2Hs would boil down to something similar but be much more restricted. Maybe you could elaborate on your idea again. I'm pretty sure I do not understand the details of your proposal. Cheers, Axel. ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Gtk2hs-devel mailing list Gtk2hs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gtk2hs-devel
