Re: concurrency guarentees clarification
John Meacham [EMAIL PROTECTED] writes: * Foreign concurrent calls, handle IO, and all IO actions that directly interact with the world outside the current process all must be yield-points. (in addition to any yield-points implied by the progress guarentee) If an IO call includes a long period of waiting, we don't only want it to yield before or after it. We want it to allow other threads to proceed during the whole wait. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: unsafePerformIO and cooperative concurrency
Ashley Yakeley [EMAIL PROTECTED] writes: Is there a ticket for this? I would prefer that unsafePerformIO and friends not be part of the standard. I would prefer otherwise. Every implementation supports it, which proves that it's useful. And it's no less unsafe than FFI. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: FFI, safe vs unsafe
John Meacham [EMAIL PROTECTED] writes: Checking thread local state for _every_ foregin call is definitly not an option either. (but for specificially annotated ones it is fine.) BTW, does Haskell support foreign code calling Haskell in a thread which the Haskell runtime has not seen before? Does it work in GHC? If so, does it show the same ThreadId from that point until OS thread's death (like in Kogut), or a new ThreadId for each callback (like in Python)? -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: FFI, safe vs unsafe
John Meacham [EMAIL PROTECTED] writes: I object to the idea that concurrent calls are 'safer'. getting it wrong either way is a bug. it should fail in the most obvious way rather than the way that can remain hidden for a long time. I wouldn't consider it a bug of an implementation if it makes a call behave like concurrent when it's specified as non-concurrent. If a library wants to make it a critical section, it should use a mutex (MVar). Or there should be another kind of foreign call which requires serialization of calls. But of which calls? it's rarely the case that it needs to be serialized with other calls to the same function only, and also rare that it must be serialized with everything else, so the granularity of the mutex must be explicit. It's fine to code the mutex explicitly if there is a kosher way to make it global. Non-concurrent calls which really blocks other thread should be treated only as an efficiency trick, as in implementations where the runtime is non-reentrant and dispatches threads running Haskell code internally, making such call without ensuring that other Haskell threads have other OS threads to run them is faster. OTOH in implementations which run Haskell threads truly in parallel, the natural default is to let C code behave concurrently. Ensuring that it is serialized would require extra work which is counter-productive. For functions like sqrt() the programmer wants to say that there is no need to make it concurrent, without also saying that it requires calls to be serialized. Which is why I'd prefer some term involving 'blocking' because that is the issue. blocking calls are exactly those you need to make concurrent in order to ensure the progress guarentee. What about getaddrinfo()? It doesn't synchronize with the rest of the program, it will eventually complete no matter whether other threads make progress, so making it concurrent is not necessary for correctness. It should be made concurrent nevertheless because it might take a long time. It does block; if it didn't block but needed the same time for an internal computation which doesn't go back to Haskell, it would still benefit from making the call concurrent. It is true that concurrent calls often coincide with blocking. It's simply the most common reason for a single non-calling-back function to take a long time, and one which can often be predicted statically (operations of extremely long integers might take a long time too, but it would be hard to differentiate them from the most which don't). The name 'concurrent' would be fine with me if the default is 'not necessarily concurrent'. If concurrent calls are the default, the name 'nonconcurrent' is not so good, because it would seem to imply some serialization which is not mandatory. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Signals + minimal proposal
Simon Marlow [EMAIL PROTECTED] writes: How does cancelling a thread differ from sending it an exception? It doesn't. By cancelling I mean just sending a particular async exception. Can a thread be GC'd without being sent an exception first? Yes, but I'm now changing this. Unfortunately it doesn't seem possible to guarantee proper stack unwinding in all cases: POSIX threads evaporate after fork() in the child process. This means that foreign code performing callbacks doesn't exist there, except in the thread doing the fork; the C stacks are inaccessible. So in Kogut in this case bound threads become unbound, and they only run up to the end of the nearest callback from foreign code: then they are killed immediately. Another technical limitation: in a build where OS threads are not used, callbacks returning in a non-LIFO order must wait for the callback using the top of the OS stack to return. It's impossible to cause such thread to continue immediately even if it gets an exception. There are also limitations caused by principles I've adopted myself. I have scoped mutex locking and scoped unlocking. In particular waiting for a condition unlocks the mutex and always relocks it before returning. When a thread is waiting to relock a mutex when exiting a scope (rather than when entering a scope), it must absolutely lock it before it can continue, in order to guarantee consistent state of the mutex in regions of code. So I'm going to send threads about to be GC'd a signal rather than an exception; it will be handled only if the thread has signals unblocked. There is another case similar to GC'ing a thread: when the runtime discovers that there are no threads to be run, to wait for I/O, to wait for a timeout, and the thread handling system signals doesn't seem to be intentionally waiting for signals (it's not blocked on a Kogut construct similar to POSIX sigwait), the runtime attempts to wake up the thread handling system signals with a Deadlock signal, so the program can react to a total deadlock. Of course a deadlock of only a subset of threads won't be detected if the threads are not GC'd. When waiting for the rest of threads at program exit, it might happen that some threads won't want to return after being cancelled, e.g. they have signals blocked or they lock up during cleanup. Such case would normally be a deadlock (the main thread is waiting until they finish, and they are waiting for something else), but the above mechanism causes the main thread to be woken up and continue even though some threads have not finished. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Signals + minimal proposal
John Meacham [EMAIL PROTECTED] writes: forkIO action = forkIO' action' where action' = do myThreadId = onExit . throwTo PleaseExit action This would be a memory leak: even after the thread finishes, its onExit handler would remain registered. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: asynchronous exceptions
Simon Marlow [EMAIL PROTECTED] writes: BTW, I just realised a better way to express block. If block is supposed to count nesting, then we have a problem that you can still unblock exceptions even within a block by using sufficient number of unblocks, so the right way is to give block this type: block :: ((IO b - IO b) - IO a) - IO a Or perhaps: block :: ((forall b. IO b - IO b) - IO a) - IO a It doesn't fit classic mutexes and conditions with my semantics of implicit blocking, because condition wait should restore the blocking state to the one before taking the mutex - but the condition wait is written without explicit passing of any value from the point of taking the mutex. I'm not sure how it carries over to Haskell patterns though. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: asynchronous exceptions
Simon Marlow [EMAIL PROTECTED] writes: I think it's unnecessary to treat signals in the way you do - you're assuming that a signal interrupts the current thread and runs a new computation (the signal handler) on the same stack, completely blocking the interrupted thread until the signal handler completes. This is the wrong way to view signal handlers, IMO: they should run in completely separate threads (perhaps a higher priority thread, if possible). This can be emulated in my model: by designating a thread for system signals, possibly even letting it spawn a new thread for each signal. Most Unix signals are supposed to abort the process however, and thus a mechanism for aborting one thread from another is needed anyway. I think async exceptions are not that much easier than async signals. Async signals include the ability to pause threads in safe points, which is needed for SIGSTOP / SIGTSTP and for my fork() wrapper. This is not archievable with signals spawning threads + async exceptions. + you don't have to block signals just because you happen to be holding a mutex. Synchronisation with a signal handler is just synchronisation with another thread. It's still very probable that taking a mutex coincides with the need to block async exceptions: an async exception in the middle of a critical section implies a danger of leaving data in an inconsistent state. Reasons for automatic blocking of async signals carry over to async exceptions. I agree with your assessment of the problems with interruptible operations in GHC: that it is impossible to completely block async exceptions across a computation. We could certainly add a way to do this. Is that the substance of your objection to GHC's async exception mechanism? Regarding interruptible operations, this and one more thing that I haven't written there: The fact that some function uses an interruptible operation internally is a visible aspect of its behavior, both in GHC design and in mine. This means that choosing which operations are interruptible should be done carefully: even if some operation blocks the thread, it might be a bad choice for an interruption point, because usage of some blocking operations should better not have to be exposed. In my case such blocking but uninterruptible operations include waiting for a mutex, and waiting for a lazy variable, among others. But Concurrent Haskell uses a single construct of MVars as mutexes, semaphores, or communication channels. The runtime can't recognize the pattern of usage to distinguish these cases. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: important news: refocusing discussion
Simon Marlow [EMAIL PROTECTED] writes: I think it would be a mistake to relegate concurrency to an addendum; it is a central feature of the language, and in fact is one area where Haskell (strictly speaking GHC) is really beginning to demonstrate significant advantages over other languages. We should make the most of it. I agree. Concurrency is needed for finalizers (except those which only call system functions, without mutating other objects). -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: asynchronous exceptions
Simon Marlow [EMAIL PROTECTED] writes: I'm not sure whether asynchronous exceptions should be in Haskell'. I don't feel entirely comfortable about the interruptible operations facet of the design, I designed that differently for my language. There is a distinct synchronous mode where asynchronous exceptions are handled by certain operations only, similarly to POSIX deferred mode of thread cancellation. This allows to use blocking operations without being interrupted. http://www.cs.ioc.ee/tfp-icfp-gpce05/tfp-proc/06num.pdf Actually I support asynchronous signals, not just exceptions: the reaction to a signal can be something other than throwing an exception. For example Linux SIGWINCH should be handled by resizing and repainting the screen at an appropriate moment, not by aborting any computation in progress. The fact that throwTo can interrupt a takeMVar, but can't interrupt a foreign call, even a concurrent one, is a bit strange. When entering foreign mode in C code embedded in Kogut (which corresponds to concurrent foreign imports in Haskell but is less convenient to use), it's possible to specify how that thread wishes to be interrupted in case someone else sends it an asynchronous signal during the foreign trip. The only implemented application of this mechanism is sending a Unix signal. This is enough to interrupt blocking syscalls like waitpid. If waitpid fails and errno == EINTR, pending signals for this thread are processed and waiting continues (unless some signal handler has thrown an exception). Implementing this without race conditions requires a sigsafe library or something equivalent. John Meacham [EMAIL PROTECTED] writes: * do we require the thrower to 'block' until the signal is recieved? (only relevant to pre-emptive implementations) My language doesn't do it, and I'm not convinced that Haskell should block. It's more efficient to make this non-blocking, and I think usually this is what is needed. * what happens if mutilple thrown exceptions pile up before the catcher gets to them? In my language each thread has a queue of pending asynchronous signals, and they are processed in order. Handling an asynchronous signal, or throwing an exception until it is handled, blocks further signals automatically, so more signals are processed only after the previous signal was handled. An exception handler is not in a tail position wrt. the catching construct, for two reasons: the state of asynchronous signals is restored after handling the exception, and a stack trace shown when the exception is propagated to the toplevel without being handled includes code in unfinished exception handlers. There is a separate exception handling syntax when the exception should be considered already handled, for cases when the exception handler should be in a tail context. * what happns to exceptions that fall off the end of threads, or the main thread? (should be answered anyway) In my case a thread body ends with a value or with an exception, and this can be examined when joining a thread, or by default the exception is propagated in the joiner. This has a disadvantage that errors in threads nobody waits for might be left undetected, unless they use an explicit wrapper. For the main thread there is a settable handler of exceptions reaching the toplevel, which by default handles some exceptions specially (Unix signals, and a request of program termination), and others are printed along with a stack trace. * promtness? how much work is the target allowed to do before it sees the exception? pthreads allows an implementation to delay processing an exception to a cancellation point do we want the same thing in haskell? Perhaps. My design includes that. David Roundy [EMAIL PROTECTED] writes: It would also be nice to address signal behavior, and by default state that signals should be converted to asynchronous exceptions. This is not enough for SIGWINCH, or for SIGHUP used to trigger reloading configuration files. OTOH purity of Haskell's functional subsystem has some nice consequences for asynchronous exceptions which don't have to carry over to asynchronous signals which don't necessarily abort the computation. If the signal is guaranteed to abort some part of IO code, then it makes sense to revert thunks under evaluation. If the signal only causes to execute some handler, then reverting them might be wasteful, as they will soon be needed again. The only downside I can see of this as default behavior would be that in cooperative systems the response to a sigTERM might be very slow. Right, it's a pity, and I agree that benefits outweigh this problem. In my implementation the thread which handles system signals (settable, defaults to the main thread) needs to be chosen by the scheduler in order to process the signal. It might take some time if there is a lot of threads. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^
Re: Export lists in modules
Simon Marlow [EMAIL PROTECTED] writes: The main difference is that I'm doing away with parentheses, commas, and export specifiers, and using layout and full declaration syntax instead. (I don't really want to discuss this very rough idea any more though, it's just a distraction, and I'm not sure I like it anyway). I like this general idea, I was thinking about something similar a long time ago. But in case of a large datatype, e.g. an AST, we certainly don't want to duplicate it in whole. It should be sufficient in the export section. So perhaps what is really needed is the ability to split a module into a public part and a private part, and allowing to duplicate certain definitions in a partial form, e.g. have both 'data Foo' and 'data Foo = Con1 | Con2' in the same module. The details are not obvious though because the syntax has not been designed with this in mind. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: FilePath as ADT
Axel Simon [EMAIL PROTECTED] writes: The solution of representing a file name abstractly is also used by the Java libraries. I think it is not. Besides using Java UTF-16 strings for filenames, there is the File class, but it also uses Java strings. The documentation of listFiles() says that each resulting File is made using the File(File, String) constructor. The GNU Java implementation uses a single Java string inside it. On Windows the OS uses UTF-16 strings natively rather than byte sequences. UTF-16 and Unicode is almost interconvertible (modulo illegal sequences of surrogates), while converting between UTF-16 and byte sequences is messy. This means that unconditionally using Word8 as the representation of filenames would be bad. I don't know a good solution. * * * Encouraged by Mono, for my language Kogut I adopted a hack that Unicode people hate: the possibility to use a modified UTF-8 variant where byte sequences which are illegal in UTF-8 are decoded into U+ followed by another character. This encoding is used as the default encoding instead of the true UTF-8 if the locale says that UTF-8 should be used and a particular environment variable is set (KO_UTF8_ESCAPED_BYTES=1). The encoding has the following properties: - Any byte sequence is decodable to a character sequence, which encodes back to the original byte sequence. - Different character sequences encode to different byte sequences (the U+ escape is valid only when it would be necessary). - It coincides with UTF-8 for valid UTF-8 byte sequences not containing 0x00, and character sequences not containing U+. It's a hack, and doesn't address other encodings than UTF-8, but it was good enough for me; it allows to maintain the illusion that OS strings are character strings. Alternatives were: * Use byte strings and character strings in different places, sometimes using a different type depending on the OS (Windows filenames would be character strings). Disadvantages: It's hard to write a filename to a text file. The API is more complex. The programmer must too often care about the kind of a string. * Fail when encountering byte strings which can't be decoded. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: give equal rights to types and classes! :)
Bulat Ziganshin [EMAIL PROTECTED] writes: if my idea was incorporated in Haskell, this change don't require even changing signatures of most functions working with arrays - just Array type become Array interface, what a much difference? What would 'Eq - Eq - Ord - Bool' mean? '(Eq a, Eq b, Ord c) = a - b - c - Bool'? '(Eq a, Ord b) = a - a - b - Bool'? '(Eq a, Ord a) = a - a - a - Bool'? -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Test performance impact
Simon Marlow [EMAIL PROTECTED] writes: Summary: 2 programs failed to compile due to type errors (anna, gg). One program did 19% more allocation, a few other programs increased allocation very slightly (2%). I wonder how many programs would fail to compile if local identifier bindings without type signatures would be treated as pattern bindings (monomorphic, no matter whether overloaded or not) and global ones as polymorphic. This is the solution I would make. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
