> Michael Hobbs <[EMAIL PROTECTED]> and I were just discussing
> the use of GHC's threads (ie, the `Concurrent' library)
> within GTK+ applications when we came across an
> inconsistency between ``The STG runtime system (revised)''
> and the actual implementation of the RTS in GHC 4.02. The
> specification says in Section 4.6 (about "safe" and "unsafe"
> C calls) that in case of a "safe" call, a new OS thread is
> spawned to execute the called C code - I also remember Simon
> (PJ) mentioning something similar at last years IFL.
> However, on inspection of the actual implementation, we
> couldn't find any use of OS threads. Maybe this is just a
> matter of the implementation being behind the specification,
> but in would be interesting to know how things are expected
> to develop.
Yep, that's right. We haven't got around to implementing C-calls using OS
threads yet, but it's certainly in the queue.
A bit of background: the real difference between "safe" (_ccall_GC) and
"unsafe" (_ccall_) C-calls is that "safe" calls are allowed to re-enter the
GHC RTS, to perform a GC or start new Haskell threads via evalIO() or
similar. In addition, "safe" C-calls are intended to guard against blockage
in the C-call by using OS threads (but this part isn't implemented yet) (*).
If you don't need to do any of this, then "unsafe" C-calls are acceptable.
The reason we make the distinction is that a safe C-call must tidy away the
Hsakell thread state in case of RTS re-entry or GC striking. This takes
quite a few instructions, and you don't need it most of the time.
(*) there's another problem with RTS re-entry if we don't use OS threads: if
the RTS is re-entered to run a new Haskell thread, then any previous calls
to evalIO() can't return until the current call returns, even if the Haskell
thread finished long ago. This can cause deadlock in certain situations
(but you have to have a pretty twisted mind to conjure up an example :-).
> This is interesting because GTK+ is not thread-safe (as far
> as I know, partially because X is not thread-safe and GTK+
> makes no attempt at guarding its X calls against re-entry).[1]
> However, as long as GHC's RTS *does not* spawn an OS thread
> for safe FFI calls, there is not problem, because GHC's
> scheduler does not really interrupt Haskell threads, but
> just asks them to yield by setting a flag (and this can
> barely interrupt GTK+).
>
> If the RTS, however, starts to use OS threads for safe FFI
> calls, GTK+ may end up in a different thread as the Haskell
> scheduler, which in turn may run Haskell threads that again
> call GTK+, thus causing disaster. One way to prevent such
> disaster (without assigning all GTK+ interaction of an
> application to a dedicated thread) would be to have some new
> synchronization primitives in the RTS (and the required API
> in `Concurrent'). For example, to know when all threads
> created by a callback have finished.
A good point - if we use OS threads to implement safe C-calls, then you can
only call thread-safe external libraries. There are thread-safe versions of
the X11 libraries, as I recall. A first-approximation solution for making a
library thread safe is to wrap it in a giant-lock semaphore, so only one OS
thread has access to any part of the library at a time, but I agree that
seems a bit excessive if you didn't want threads in the first place.
> The required synchronization could of course be programmed
> using `Concurrent's semaphores, but with the disadvantage
> that if we want to provide this thread-safety transparently
> for user applications in a library, we have to define a
> "new" IO monad on top of the standard one (to thread the
> reference to the semaphore to all library calls). As a
> consequence, all standard IO operations would have to be
> lifted to the new IO monad.[2]
Sven Panne pointed out that you can use unsafePerformIO to avoid plumming
the semaphore through the whole program. That's what I'd do, and we've used
similar tricks in GHC to avoid excessive plumming.
> Finally, I am wondering how big the performance impact of
> using a separate IO thread per safe FFI call is. For
> example, in the GTK+ binding I often cannot use unsafe
> calls, because the calls can trigger callbacks (ie, may
> reenter Haskell), but I know that they cannot block. On the
> other hand, the usage of OS threads is nice when calling an
> external routine that can block. The question, however, is
> whether we want to pay for an OS thread each time a call can
> potentially re-enter the Haskell scheduler. Preforking
> OS threads can make things more efficient, but I am still not
> sure whether it wouldn't be a serious overhead. Maybe we
> need three types of calls: unsafe, safe, and supersafe ;-)
Maybe indeed. Bear in mind the deadlock problem described above too - you
might need supersafe C-calls even if the C-call can't block, to avoid
Haskell deadlocks.
> Let's summarize the questions:
>
> * Will safe FFI calls eventually be executed by extra OS
> threads?
Yes, in one form or another.
> * Would you support additional synchronization primitives in
> the RTS (and `Concurrent') to support making the use of
> libraries like GTK+ thread-safe?
Suggest an interface, and we'll add 'em! I'd much prefer to do this stuff
using the primitives we already have, though. Since I recently added
asynchronous exceptions and the ability to raise exceptions in other
threads, many of the thread-administration problems can be solved without
adding extra primitives.
> * How about three kinds of FFI calls?
A distinct probability.
Cheers,
Simon
