Re: Concurrency
Hello William, Sunday, October 15, 2006, 5:07:26 PM, you wrote: http://www.seas.upenn.edu/~lipeng/homepage/unify.html can this be ported to windows? (i don't yet read the paper) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-prime mailing list Haskell-prime@haskell.org http://www.haskell.org/mailman/listinfo/haskell-prime
Re: concurrency guarentees clarification
John Meacham wrote: * every runnable thread is guarenteed to run in a finite amount of time if a program reaches a yield-point infinitly often. What happens if one of the thread ends up in an infinite loop that contains a yield point? Infinitely often is unclear (I think I know what you're trying to say, but this is because I think I know what you're trying to say overall, and not because of these words). I'd say something like if, after hitting a yield point, the program hits another yield point in a finite amount of time (the start of execution and program termination being considered yield points for the purposes of this rule). ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: concurrency guarentees clarification
John Meacham [EMAIL PROTECTED] writes: * Foreign concurrent calls, handle IO, and all IO actions that directly interact with the world outside the current process all must be yield-points. (in addition to any yield-points implied by the progress guarentee) If an IO call includes a long period of waiting, we don't only want it to yield before or after it. We want it to allow other threads to proceed during the whole wait. -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: Concurrency, FFI status
On 22 April 2006 04:03, John Meacham wrote: On Fri, Apr 21, 2006 at 10:01:51AM -0400, Manuel M T Chakravarty wrote: Concerning the issue of preemptive versus cooperative concurrency, I still think cooperative concurrency is pretty useless. Is there any non-toy application that actually uses Hugs' current cooperative concurrency? A couple of notes * How many non-toy applications can use hugs at all independent of concurrency? in a big concurrent app you are most likely going to need some ghc extension anyway. * It is unclear that pre-emptiveness even buys you much, standardwise. the only thing it can give over a cooperative one is better latency and even that can't be done as oleg pointed out without careful control of when lazy (pure) code gets evaluated, a similar situation to what one has to think about with cooperative implementations. I disagree that these two problems are equivalent in difficulty. In a preemptive implementation, I can execute any pure non-IO code in a thread without regard for how this affects the responsiveness of other threads. In a cooperative system, however, the programmer has to understand the performance characteristics of all the non-IO computations in the program, and if any might affect the latency beyond what is acceptable the entire code path has to be IO-monadised, or split up into sub-computations so that yields can be inserted. This affects the *entire program*. Including libraries that you didn't write and can't change. With preemptive concurrency, you just don't have to care about latency, it is under the control of the scheduler. The problem you are claiming is similar is the need for the programmer to understand the performance of non-IO code that executes while a resource is being held, such as an MVar. This happens much less frequently, it is a small fraction of the entire program. True it might happen in library code that you don't control, but you can legitimately claim that as a bug in the library, whereas claiming that a library of non-IO code is buggy because it is too slow and doesn't yield is unreasonable. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency, FFI status
Simon Marlow: I have now summarised the concurrency proposal status, here: http://hackage.haskell.org/cgi-bin/haskell-prime/trac.cgi/wiki/Concurren cy I have tried to summarise the various points that have arisen during the discussion. If anyone feels they have been mis-paraphrased, or I have forgotten something, please feel free to edit, or send me some text for inclusion. I don't want to include long gobs of text in here, though: just summarise the main points, and if necessary link to relevant mailing list posts. Good summary. Concerning the issue of preemptive versus cooperative concurrency, I still think cooperative concurrency is pretty useless. Is there any non-toy application that actually uses Hugs' current cooperative concurrency? Concerning the trouble of Hugs and Jhc to implement preemptive concurrency, IMHO that's a significant design flaw in these implementations. Preemptive concurrency is important for many applications and, if anything, will become more important with new architectures. Fundamental limits on being able to support this, fundamentally limit the application space. I'd rather not see that design flaw being transferred from these implementations to the language standard. Manuel ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency, FFI status
On Fri, Apr 21, 2006 at 10:01:51AM -0400, Manuel M T Chakravarty wrote: Concerning the issue of preemptive versus cooperative concurrency, I still think cooperative concurrency is pretty useless. Is there any non-toy application that actually uses Hugs' current cooperative concurrency? A couple of notes * How many non-toy applications can use hugs at all independent of concurrency? in a big concurrent app you are most likely going to need some ghc extension anyway. * It is unclear that pre-emptiveness even buys you much, standardwise. the only thing it can give over a cooperative one is better latency and even that can't be done as oleg pointed out without careful control of when lazy (pure) code gets evaluated, a similar situation to what one has to think about with cooperative implementations. * Hugs does not implement concurrency. A couple tests show that. run1 = print foo run1 run2 = getLine = print run2 main = do forkIO run1 run2 this should continually print foo while waiting for input on any haskell-prime implementation due to the progress guarentee. it does not on hugs, this makes hugs concurrency not really concurrency IMHO, more like explicit coroutines with an odd interface. (which is the base of a good cooperative concurrent implementation, but not the same thing) Concerning the trouble of Hugs and Jhc to implement preemptive concurrency, IMHO that's a significant design flaw in these implementations. Preemptive concurrency is important for many applications and, if anything, will become more important with new architectures. Fundamental limits on being able to support this, fundamentally limit the application space. I'd rather not see that design flaw being transferred from these implementations to the language standard. It is not a design flaw, it is a choice. pre-emptiveness is not worth the effort given haskells other properties for many implementation models. It buys you very little, but implementing it can cause signifigant run-time overheads compared to cooperative ones for code that doesn't even use concurrency. I don't care about the difficulty of implementation, I care about the generated code and the ability to write standards compliant _and_ efficient haskell prime implementations. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: Concurrency, FFI status
Good summary. I have made a few edits mainly to clarify what (I think) is being said. Under cooperative or preemptive concurrency I'd like someone two write down as precisely as possible what it means to say the spec requires cooperative concurrency or the spec requires preemptive concurrency. That would set the context for the following choices Simon | -Original Message- | From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of | Simon Marlow | Sent: 13 April 2006 14:42 | To: haskell-prime@haskell.org | Subject: Concurrency, FFI status | | I have now summarised the concurrency proposal status, here: | | | http://hackage.haskell.org/cgi-bin/haskell-prime/trac.cgi/wiki/Concurren | cy | | I have tried to summarise the various points that have arisen during the | discussion. If anyone feels they have been mis-paraphrased, or I have | forgotten something, please feel free to edit, or send me some text for | inclusion. I don't want to include long gobs of text in here, though: | just summarise the main points, and if necessary link to relevant | mailing list posts. | | Cheers, | Simon | ___ | Haskell-prime mailing list | Haskell-prime@haskell.org | http://haskell.org/mailman/listinfo/haskell-prime ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: asynchronous exceptions (was: RE: Concurrency)
On Tue, Apr 04, 2006 at 01:33:39PM +0100, Simon Marlow wrote: I'm not sure whether asynchronous exceptions should be in Haskell'. I don't feel entirely comfortable about the interruptible operations facet of the design, and I'm hoping that STM can clean things up: after all, STM already gives you a much nicer way to program in an exception-safe way, as long as you aren't doing any real I/O. For me, asynchronous exceptions are the primary reason to use concurrent Haskell. They're the only way I'm aware of to write a program that handles signals in Haskell, and it's be a real shame to leave Haskell' programs unable to handle signals--it means that any real-world programs that deal with locking or the like will need to use non-standard extensions. Unless you can come up with some other way to deal with signals. Having no chance to clean up when control-C is hit isn't an acceptable alternative, and neither is simply ignoring control-C and forcing users to run kill (and then once again get no chance to clean up!). Another option would be to hard-code signals into some sort of ordinary exceptions, but that's not very good, since you'd then still want to split exceptions into two camps, so that you could run a catch that only catches signals generated *by* the IO action that you're running (i.e. you often want to remove a file, and ignore any sort of failure, but don't want to accidentally ignore a sigTERM that arrives during this process). I suppose you could do this with the complicated catchJust, but that's a pain. It's nice having a small (and documented) set of exceptions that most IO operations can throw. The fact that throwTo can interrupt a takeMVar, but can't interrupt a foreign call, even a concurrent one, is a bit strange. We have this odd situation in GHC right now where throwTo can interrupt threadDelay on Unix, but not on Windows because threadDelay maps directly to a foreign call to Sleep() on Windows. To fix this I have to implement the I/O manager thread on Windows (I should do this anyway, though). [...] The only guarantee you can give is the exception isn't delayed indefinitely, unless the target thread remains inside a block. Just like the fairness property for MVars. I think this is fine. There's no need for strong guarantees that asynchronous exceptions are delivered soon or interrrupt any particular external function calls, or even interrupt particular standard library functions. At least to me, that's what makes them asynchronous. In other words, I wouldn't mind cooperative asynchronous function calls. Which is to say, that I don't see any reason the standard IO library shouldn't be allowed (by the standard) to use block in all its calls. This does limit the power of their application to signal-handling, but if you really want signal-catching in ffi functions, those functions could install their own signal-catchers. My main concern is that as far as I can see, without asynchronous exceptions there's no way to implement this sort of functionality in pure Haskell. Actually, I suppose you could do this with a (cooperative) implementation of asynchronous exceptions using just MVars and concurrency by rewriting all the IO calls you use to first check whether an asynchronous exception has been thrown, but rewriting all the std library functions seems like a rather crude way of doing this. On the other hand, I suppose that this could also provide a reference implementation of asynchronous exceptions for any Haskell' that supports concurrency... -- David Roundy http://www.darcs.net ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: asynchronous exceptions (was: RE: Concurrency)
On 05 April 2006 12:47, David Roundy wrote: On Tue, Apr 04, 2006 at 01:33:39PM +0100, Simon Marlow wrote: The fact that throwTo can interrupt a takeMVar, but can't interrupt a foreign call, even a concurrent one, is a bit strange. We have this odd situation in GHC right now where throwTo can interrupt threadDelay on Unix, but not on Windows because threadDelay maps directly to a foreign call to Sleep() on Windows. To fix this I have to implement the I/O manager thread on Windows (I should do this anyway, though). [...] The only guarantee you can give is the exception isn't delayed indefinitely, unless the target thread remains inside a block. Just like the fairness property for MVars. I think this is fine. There's no need for strong guarantees that asynchronous exceptions are delivered soon or interrrupt any particular external function calls, or even interrupt particular standard library functions. At least to me, that's what makes them asynchronous. In other words, I wouldn't mind cooperative asynchronous function calls. Which is to say, that I don't see any reason the standard IO library shouldn't be allowed (by the standard) to use block in all its calls. Yes, maybe it's ok. The nice thing about the way block works right now is that it doesn't quite block all asynchronous exceptions, it actually turns them into synchronous exceptions which are much more tractable. For example, in GHC's IO library I couldn't face the prospect of scrutinising every line of code for exception-safety, so I just put a block around everything that modifies Handle state. This isn't nearly as bad as it sounds, because any operation that blocks inside an I/O operation is still interruptible, but because we know which operations those are, we can be prepared to handle the exceptions. In fact, since the operations that block are mostly the I/O operations themselves, we're already well prepared for handling exceptions there anyway. So it all works out nicely. Concurrency abstractions built from MVars suffer a bit from async-exception-safety. The right way is to build these abstractions using the exception-safe withMVar modifyMVar family, but they impose an overhead. Fortunately STM is exactly the right solution here. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: asynchronous exceptions (was: RE: Concurrency)
On 05 April 2006 13:38, John Meacham wrote: On Wed, Apr 05, 2006 at 07:47:08AM -0400, David Roundy wrote: For me, asynchronous exceptions are the primary reason to use concurrent Haskell. They're the only way I'm aware of to write a program that handles signals in Haskell, and it's be a real shame to leave Haskell' programs unable to handle signals--it means that any real-world programs that deal with locking or the like will need to use non-standard extensions. Unless you can come up with some other way to deal with signals. Having no chance to clean up when control-C is hit isn't an acceptable alternative, and neither is simply ignoring control-C and forcing users to run kill (and then once again get no chance to clean up!). I have been giving signals some thought, and resarching what other languages do, and have a semi-proposal-maybe. We should be careful here: the Haskell standard has so far remained platform-independent, and I think it would be nice to keep it that way. I'm not proposing that we ignore signals, just that we should clearly delimit the platform-specific bits, perhaps by putting signal support into an addendum. signals tend to be used for one of a couple purposes (some can fall into multiple categories): signal a synchronous exceptional event - SIGFPE, SIGPIPE, SIGILL, SIGSEGV signal an asynchronous exceptional event - SIGINT, SIGHUP (interactive) inform the app of an event it might want to take note of - SIGALRM, SIGCHLD, SIGWINCH, SIGHUP (daemon) I think it would make sense to have 3 mechanisms to cover these cases. signal a synchronous exceptional event - raise a (possibly imprecise) exception on the thread that produced the signal. GHC has no support for these right now. They're pretty tricky to handle, because the OS thread that caused the signal to be raised is stopped at some arbitrary instruction, and it would require some serious acrobatics to munge that OS thread into a state where it is possible to raise the (Haskell) exception. I do vaguely recall that people have achieved this in the past, in order to use page faults for write barriers, that sort of thing. SIGPIPE is possibly easier than the others. SIGFPE you can usually turn off in favour of exceptional values instead. signal an asynchronous exceptional event - the user should be able to choose the threads on which they wish to catch these, those that need to clean up after themselves. inform the app of an event it might want to take note of - these should run on their own thread, concurrently to all other threads GHC directly support the latter version, and you can implement the former with a little extra code and a global variable. I think it would be nice to do as you suggest and provide a way to have the async signals turn directly into exceptions. One problem, though, is that because we can't interrupt a foreign call with an async exception, ^C can be rather unresponsive. Perhaps I should look into this and see whether it would be possible in GHC for a concurrent foreign call to be interruptible; it would involve terminating the foreign call somehow (pthread_cancel?) before raising the exception. We can't do this in a bound thread, however. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
asynchronous exceptions (was: RE: Concurrency)
I'm not sure whether asynchronous exceptions should be in Haskell'. I don't feel entirely comfortable about the interruptible operations facet of the design, and I'm hoping that STM can clean things up: after all, STM already gives you a much nicer way to program in an exception-safe way, as long as you aren't doing any real I/O. The fact that throwTo can interrupt a takeMVar, but can't interrupt a foreign call, even a concurrent one, is a bit strange. We have this odd situation in GHC right now where throwTo can interrupt threadDelay on Unix, but not on Windows because threadDelay maps directly to a foreign call to Sleep() on Windows. To fix this I have to implement the I/O manager thread on Windows (I should do this anyway, though). On 03 April 2006 07:38, John Meacham wrote: however, in a preemptive one, it is less clear whether they may be delivered right away or not and what the implementation costs are, as threads arn't necessarily blocked at a point where they can happily deal with an exception. Perhaps the yhc guys have thought about the problem? In fact, GHC's SMP mode still doesn't implement throwTo, mainly because it's rather difficult and I need to put some serious thought into how to do it. there are also a few questions we would want answered for the spec * do we require the thrower to 'block' until the signal is recieved? (only relevant to pre-emptive implementations) GHC's throwTo does block until the exception is delivered. The main argument for doing it this way is that you only need an extra forkIO to get the other version. Another argument is that it gives you more guarantees: if two threads throwTo each other, only one will succeed. * what happens if mutilple thrown exceptions pile up before the catcher gets to them? They get delivered in some order (fairness probably applies in the same way to threads blocked on MVars). * what happns to exceptions that fall off the end of threads, or the main thread? (should be answered anyway) throwTo a thread that has completed is a no-op. * promtness? how much work is the target allowed to do before it sees the exception? pthreads allows an implementation to delay processing an exception to a cancellation point do we want the same thing in haskell? if not, how will this affect OS threaded implementations? The only guarantee you can give is the exception isn't delayed indefinitely, unless the target thread remains inside a block. Just like the fairness property for MVars. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: Concurrency
On 31 March 2006 22:15, John Meacham wrote: On Fri, Mar 31, 2006 at 04:21:26PM +0100, Simon Marlow wrote: Great. Apart from my misgivings about allowing cooperative scheduling at all, here's a few comments on the proposal: much much preferable to a standard that not everyone can implement. :) - I wouldn't include threadWaitRead, threadWaitWrite, or threadDelay at all. These can all be implemented using FFI, so don't belong in the concurrency library. Their presence is largely historical. They all have special implementations on a 'epoll' based system. threadDelay turns into the timeout parameter to select, waitread/write turn into the basic building blocks of your epoll wait-list. We definitly want these in the interface as primitves. Still not convinced. Most applications can use the standard IO library in a multithreaded program to get I/O multiplexing. The library might be implemented by using a clever epoll/kqueue/whatever interface underneath, but I don't see a reason to expose that as a standard library. And it's perfectly reasonable to implement concurrent IO without doing any clever epoll stuff: GHC on Windows does just that. IMHO, concurrency gives you a way to *avoid* needing an event interface in your language. In particular, foregin concurrent calls will most likely be implemented in _terms_ of threadWaitRead on cooperative systems. By all means, but that still doesn't mean that threadWaitRead needs to be in the standard. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On Tue, Apr 04, 2006 at 02:25:11PM +0100, Simon Marlow wrote: On 31 March 2006 22:15, John Meacham wrote: On Fri, Mar 31, 2006 at 04:21:26PM +0100, Simon Marlow wrote: Great. Apart from my misgivings about allowing cooperative scheduling at all, here's a few comments on the proposal: much much preferable to a standard that not everyone can implement. :) - I wouldn't include threadWaitRead, threadWaitWrite, or threadDelay at all. These can all be implemented using FFI, so don't belong in the concurrency library. Their presence is largely historical. They all have special implementations on a 'epoll' based system. threadDelay turns into the timeout parameter to select, waitread/write turn into the basic building blocks of your epoll wait-list. We definitly want these in the interface as primitves. Still not convinced. Most applications can use the standard IO library in a multithreaded program to get I/O multiplexing. The library might be implemented by using a clever epoll/kqueue/whatever interface underneath, but I don't see a reason to expose that as a standard library. And it's perfectly reasonable to implement concurrent IO without doing any clever epoll stuff: GHC on Windows does just that. IMHO, concurrency gives you a way to *avoid* needing an event interface in your language. Oh, it was always my plan to expose the epoll/Event library, it is something lacking in haskell currently which is a pain. Often, an EDSM loop is the best way to express an algorithm. Concurrency is great, but it is far from a panacea. In particular, foregin concurrent calls will most likely be implemented in _terms_ of threadWaitRead on cooperative systems. By all means, but that still doesn't mean that threadWaitRead needs to be in the standard. Here are a couple of uses. We need waiting routines for every interesting event, they are darn useful, and they admit a very optimized implementation. A C library which reads a file descriptor, calling threadWaitRead beforehand and then the call is much more efficient than calling the library directly. A consumer-producer set up, you don't want to read into core memory the producers data until you are sure you have somewhere to write it to, so you have a 'threadWaitWrite-read-write' loop. UI design, a common issue is displaying continuously updating information on the screen. You communicate with the X11 server via a pipe, so it is possible to just completely fill the X11 buffer with drawing requests and get a backlog killing your apps responsibility. also, by the time these backloged events actually get to the screeen they may be out of date and your app quickly crashes and burns if your incoming data rate exceeds the speed at which you can draw to the screen. Concurrency admits a nice solution, you have your thread that generates the interesting info, (say, reading data from a DSP), continually updating a shared location. your drawing routine is in a 'threadWaitWrite-readlocation-drawit' loop. notice that threadWaitWrite is needed because otherwise you would be forced to send outdated data to the screen if you couldn't separate what you write from when you write it. A C call that uses a file, but expects some precondition to be set before it can be run. One that I ran into is a curses library that waited for user input, but wanted the screen to be cleared before it could be run. clearing the screen right away would have left the user sitting staring at a blank screen with no instructions until they pressed a key, the right thing to do was to do a threadWaitRead, clear the screen, then call the routine. Efficient scheduling is another use. you might want to wait for several clients to become ready before deciding which one to actually service. Basically, they are needed any time the data you are processing is time-dependent. any time you need to decide what to read or write on the current conditions rather than what they were when you started the read or write. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: Concurrency
On 31 March 2006 13:41, John Meacham wrote: I have tried to summarize the current thinking into a proposal on the wiki. http://haskell.galois.com/cgi-bin/haskell-prime/trac.cgi/wiki/Concurrenc y I split it into 3 parts. the standard - all haskell' compilers must implement optional preemption - ability to preempt pure code, fairness guarentees, interleaved evaluation operators (merge, nmerge) optional OS threads - bound threads, SMP, reentrant concurrent FFI calls supported. comment or modify it at will. Great. Apart from my misgivings about allowing cooperative scheduling at all, here's a few comments on the proposal: - I wouldn't include threadWaitRead, threadWaitWrite, or threadDelay at all. These can all be implemented using FFI, so don't belong in the concurrency library. Their presence is largely historical. - yield bothers me a little. If it weren't for cooperative systems, yield would be semantically a no-op, because the no-starvation guarantee means you never need it for correctness. I think it's ok, just a bit unsettling. - In the optional OS threads section it says allows multiple haskell threads to run at once - actually you can provide all that without allowing multiple haskell threads to run at once, eg. ghc-6.4.1 with -threaded. I'll modify it. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On Fri, Mar 31, 2006 at 01:15:03PM -0800, John Meacham wrote: On Fri, Mar 31, 2006 at 04:21:26PM +0100, Simon Marlow wrote: Great. Apart from my misgivings about allowing cooperative scheduling at all, here's a few comments on the proposal: much much preferable to a standard that not everyone can implement. :) Are there potential users for the compromise interface? I had the impressions that those wanting concurrency needed the fairness guarantees. ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On Mon, Apr 03, 2006 at 11:38:08AM +0100, Ross Paterson wrote: On Fri, Mar 31, 2006 at 01:15:03PM -0800, John Meacham wrote: On Fri, Mar 31, 2006 at 04:21:26PM +0100, Simon Marlow wrote: Great. Apart from my misgivings about allowing cooperative scheduling at all, here's a few comments on the proposal: much much preferable to a standard that not everyone can implement. :) Are there potential users for the compromise interface? I had the impressions that those wanting concurrency needed the fairness guarantees. quite the opposite IMHO. I think for most uses cooperative implementations will be not just just fine, but preferable. We really shouldn't call it a compromise interface, cooperative threading is often considered superior to pthreads/pre-emptive threading for a wide variety of tasks. After a lot of experience writing pthreads code from both the OS and application side, I find I agree. cooperative state-threads should always be the way to go by default when writing new code unless you absolutely need one of the features of pre-emptive threading. the tasks for which state-threads work well for are IO bound multiplexing tasks, pthreads are better for CPU-bound tasks. However, most uses of concurrency are for programs that interact with the user or the external world, as in they wait for an event from a variety of sources and respond to it quickly. The limiting factor isn't processing power, but how fast the events come, how fast you can redraw the screen, your network speed, etc. exactly what state-threading is best at. Most CPU bound tasks tend to be batch processing type things like compilers, which don't need concurrency to begin with. some info on the advantages and tradeoffs http://state-threads.sourceforge.net/docs/st.html although written from the point of view of network servers, a lot is relevant to other fields. Ideally, I'd like to provide both in jhc. But cooperative is a whole lot of bang for the buck. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On Fri, Mar 31, 2006 at 04:21:26PM +0100, Simon Marlow wrote: Great. Apart from my misgivings about allowing cooperative scheduling at all, here's a few comments on the proposal: much much preferable to a standard that not everyone can implement. :) - I wouldn't include threadWaitRead, threadWaitWrite, or threadDelay at all. These can all be implemented using FFI, so don't belong in the concurrency library. Their presence is largely historical. They all have special implementations on a 'epoll' based system. threadDelay turns into the timeout parameter to select, waitread/write turn into the basic building blocks of your epoll wait-list. We definitly want these in the interface as primitves. In particular, foregin concurrent calls will most likely be implemented in _terms_ of threadWaitRead on cooperative systems. - yield bothers me a little. If it weren't for cooperative systems, yield would be semantically a no-op, because the no-starvation guarantee means you never need it for correctness. I think it's ok, just a bit unsettling. even pthreads provides it. noise I think you place a lot of faith in pre-emption. :) In my experience, it doesn't actually buy you a whole lot over state-threading in the non SMP case. everything would be different if we were thinking of different processes on the same computer, where you wouldn't want one buggy one interfering with others, but in general you consider a single program buggy or bug-free as a unit. /noise In any case, IO multiplexing is 90% of the uses of threading anyway, (ginsu,yi,gui apps that don't do background processing, etc...) which cooperative threading is ideal for. not that there arn't itches that only preemptive threads can scratch too. - In the optional OS threads section it says allows multiple haskell threads to run at once - actually you can provide all that without allowing multiple haskell threads to run at once, eg. ghc-6.4.1 with -threaded. I'll modify it. okay. yeah, I just sort of outlined the options figuring we would fill in the details later. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
I just realized that my mailer futzed this one and its headers don't match where it was actually sent. so if you are responding to it, the mail most likely is not going out to the list. make sure it is to haskell-prime and not hasuell-prime. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On Fri, Mar 31, 2006 at 01:51:14PM -0800, John Meacham wrote: If so, we should specify what exception is raised if, say, forkIO is called on such a system. We should also make it clear that single-threaded implementations are required of things such as MVars. Finally, we should add a function that indicates the level of threading support on the running system. routines that are guarenteed to fail becaues they arn't supported should not exist, haskell tries hard to catch so many errors at compile time, it would seem odd to delegate the error of missing a whole subsystem to run-time :) Not really. What if I'm writing a program that can take advantage of threading if it's available, but can degrade gracefully if not? Should I be forced to use something like cpphs to detect the presence of threading in advance? It would be better to detect this at runtime than fail to compile at all on a system that doesn't support threading, IMHO. -- John ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On 3/31/06, John Meacham [EMAIL PROTECTED] wrote: - I wouldn't include threadWaitRead, threadWaitWrite, or threadDelay at all. These can all be implemented using FFI, so don't belong in the concurrency library. Their presence is largely historical. They all have special implementations on a 'epoll' based system. threadDelay turns into the timeout parameter to select, waitread/write turn into the basic building blocks of your epoll wait-list. We definitly want these in the interface as primitves. And they're all a pain because they don't take sets of files, only single ones. Can we please have something like: threadWait :: Timeout - [Handle] - IO ? -- Taral [EMAIL PROTECTED] You can't prove anything. -- Gödel's Incompetence Theorem ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency
On Fri, Mar 31, 2006 at 04:29:59PM -0600, Taral wrote: On 3/31/06, John Meacham [EMAIL PROTECTED] wrote: - I wouldn't include threadWaitRead, threadWaitWrite, or threadDelay at all. These can all be implemented using FFI, so don't belong in the concurrency library. Their presence is largely historical. They all have special implementations on a 'epoll' based system. threadDelay turns into the timeout parameter to select, waitread/write turn into the basic building blocks of your epoll wait-list. We definitly want these in the interface as primitves. And they're all a pain because they don't take sets of files, only single ones. Can we please have something like: threadWait :: Timeout - [Handle] - IO ? Oh, that is definitly planned as part of an 'epoll' interface I have been calling Event. depending on the compiler, Concurrent might be implemented on top of Event or Event might be implemented on top of Concurrent :) In any case, I left it out of the proposal here because it is relatively orthogonal (from a design, not an implemenatition point of view) but I definitly think it should exist. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: concurrency (was Re: important news: refocusing discussion)
On Tue, Mar 28, 2006 at 10:49:36AM +0100, Malcolm Wallace wrote: Tomasz Zielonka [EMAIL PROTECTED] wrote: http://www.uncurry.com/repos/FakeSTM/ Perhaps it could serve as a drop-in replacement for STM in haskell compilers which don't implement STM directly. Nice idea. But your code already uses a whole heap of Haskell extensions which may or may not make it into Haskell'. monad transformer lib (requires MPTC) exceptions dynamically extensible exceptions deriving non-standard classes extended newtype deriving pattern guards You read the whole code? Wow! I myself would have trouble understanding how it does what it does now ;-) I could easily get rid of: deriving non-standard classes extended newtype deriving pattern guards I used GHC's exceptions, because I wanted my STM to handle them correctly, as in the STM paper. In a implementation without exceptions, I could probably get away with hand made exception handling. The rest would be a bit more difficult to remove, but I think it could be possible more or less elegantly. Best regards Tomasz ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency (was: RE: Re[2]: important news: refocusing discussion)
On Mon, Mar 27, 2006 at 03:36:55PM +0100, Simon Marlow wrote: But before we get carried away figuring out all the pros and cons of various options, let me point out once again that This is just a marketing decision Because (a) we're going to standardise concurrency anyway concurrency is a hugely overloaded term in this whole discussion. I am hoping to break out what it actually means on the wiki some more. (b) it is unlikely that Hugs or JHC will implement concurrency even if it goes into the standard Well, if the standard is unimplemented for certain compilers, I think we need to work on the standard because that would be a deficiency of it. I would very much like to be able to write portable concurrent programs which doesn't necessarily mean GHC style concurrency or nothing. given the choice between 1. a standard specifying something most people can't implement 2. a widely available but not mentioned in the standard extension 2 seems much more preferable and we should err on that side. of course, this is a false dichotomy as there are happy mediums in the middle I hope we can arrive at. I am thinking jhc will offer two concurrency mechanisms eventually, 1. state-thread based threading based on a portable user space library. so you get O'Haskell or hugsish concurrency by just using the right library. 2. one OS thread per haskell thread, no guarentees about repeated work between threads. the reasoning being that a programer can avoid the problem of repeated work by being clever, but the run-time cost of suspending partial evaluations and protecting in-progress computations is unavoidable. some profiling support will probably be needed to aid a programmer in determining if repeated work is an issue. I think it is very likely that hugs and jhc and yhc will all implement concurrency of some sort so it would be odd if ghc's happened to be the only one that is standards compliant by definition. (c) the question is just whether the brand Haskell' encompasses concurrency or not (I thought I'd use that word because I know JL likes it so much :-) I don't think it should necessarily, at least not a type of concurrency that can't be widely implemented. it would be bad for the brand and sort of negate one of the points of haskell' if GHC were the only true implementation. Yes there are several ramifications of this decision, but none of them are technical. As I see it, we either specify Concurrency as an addendum, or NoConcurrency as an addendum, and both options are about the same amount of work. this is a big big understatement. the concurrency specifications are completly underspecified and there is a lot of technical work that would be needed to bring them up to snuff. the current proposal basically says do what GHC does in a lot of words. So on that note, I'll restate my preference that Haskell' should include concurrency, and leave it at that. We can start the standardisation process without arriving at a conclusion on this issue anyway. indeed. but I feel that just saying GHC style or nothing would sort of suck as there are very fruitful intermediate options without the caveats of the ghc model. I think we actually are going to need to dig into the details of concurrency, one way or another and I'd like to see something portable/good/and uncompromising come out of the commitee if it exists. If we are going to add concurrency, I'd like to see it done right. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
RE: Concurrency (was: RE: Re[2]: important news: refocusingdiscussion)
On 29 March 2006 10:16, John Meacham wrote: On Mon, Mar 27, 2006 at 03:36:55PM +0100, Simon Marlow wrote: (b) it is unlikely that Hugs or JHC will implement concurrency even if it goes into the standard Well, if the standard is unimplemented for certain compilers, I think we need to work on the standard because that would be a deficiency of it. I would very much like to be able to write portable concurrent programs which doesn't necessarily mean GHC style concurrency or nothing. given the choice between 1. a standard specifying something most people can't implement 2. a widely available but not mentioned in the standard extension 2 seems much more preferable and we should err on that side. Fair enough - I take that as a vote for a concurrency addendum. I think it's a bit unfair to talk about GHC-style concurrency. There are many different ways to implement exactly what GHC currently provides. In fact, we were very careful when designing it to ensure that this was the case: 1. one OS thread per Haskell thread, giant lock around the runtime 2. one OS thread for the runtime, fork OS threads for foreign calls 3. hybrid implementation with a pool of OS threads (GHC's implementation). You even mention another implementation that we hadn't considered: 4. one OS thread per Haskell thread but no lock around the runtime, with possibly repeated work. YHC uses (2), for reference. You're proposing (4) as one possibility for JHC. Cheers, Simon ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency (was: RE: Re[2]: important news: refocusingdiscussion)
On Wed, Mar 29, 2006 at 11:56:41AM +0100, Simon Marlow wrote: Fair enough - I take that as a vote for a concurrency addendum. Actually, I think there is a lot we can standardize in a portable way when it comes to concurrency without compromising the ability for any compiler to implement it and I think it would be very worthwhile to do so. in the report proper. I think it's a bit unfair to talk about GHC-style concurrency. There are many different ways to implement exactly what GHC currently provides. In fact, we were very careful when designing it to ensure that this was the case: yeah, when I say GHC style concurrency, I mean the interface that ghc has. forkIO,MVar, etc... as opposed to event-loop, O'Haskell, expliticly scheduled, manual continuations, etc.. but I have been clumsy about whether I mean cooperative or fully-preemptive by GHC-style. I'll try to make that clear. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency (was: RE: Re[2]: important news: refocusingdiscussion)
John Meacham: On Wed, Mar 29, 2006 at 11:56:41AM +0100, Simon Marlow wrote: Fair enough - I take that as a vote for a concurrency addendum. Actually, I think there is a lot we can standardize in a portable way when it comes to concurrency without compromising the ability for any compiler to implement it and I think it would be very worthwhile to do so. in the report proper. I think it's a bit unfair to talk about GHC-style concurrency. There are many different ways to implement exactly what GHC currently provides. In fact, we were very careful when designing it to ensure that this was the case: yeah, when I say GHC style concurrency, I mean the interface that ghc has. forkIO,MVar, etc... as opposed to event-loop, O'Haskell, expliticly scheduled, manual continuations, etc.. As I see it, it's really only GHC's API which is up for discussion for inclusion in Haskell', as we we decided that we largely want to go with already implemented and tested approaches. Manuel ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: Concurrency (was: RE: Re[2]: important news: refocusing discussion)
Simon Marlow [EMAIL PROTECTED] wrote: (a) we're going to standardise concurrency anyway Well, but that only begs the question, what *kind* of concurrency are we going to standardise on? e.g. Will we admit all variations of scheduling (co-operative, time-slice, and pre-emptive)? (b) it is unlikely that Hugs or JHC will implement concurrency even if it goes into the standard Now this is something that puzzles me. I was under the impression that Hugs already implements concurrency, using pretty much the same APIs as ghc. I'd also like to know a bit more about jhc's position here. Is it just that JohnM wants to keep his compiler pure and free from having a runtime-system? Or are there other issues? Yes there are several ramifications of this decision, but none of them are technical. As I see it, we either specify Concurrency as an addendum, or NoConcurrency as an addendum, and both options are about the same amount of work. There are certainly technical questions. If Hugs's implementation of concurrency is not concurrency after all, on what basis do we make that determination? Why is a definition of concurrency that encompasses both ghc and Hugs models unacceptable? Regards, Malcolm ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: concurrency (was Re: important news: refocusing discussion)
On Tue, 2006-03-28 at 11:05 +0100, Malcolm Wallace wrote: (snip) * IORef is inherently thread-unsafe, and so we should eliminate IORefs from the language. That's not quite true, as you can have an IORef guarded by an MVar. Why would you want such a thing? For instance, you might write a library with two IORefs and one MVar instead of two MVars in order to reduce the possibility of deadlock. Is it the case that a library is thread-safe as long as it doesn't use IORefs, though? I trolled around base looking for libraries that might not be thread-safe and found only that HashTable uses an IORef, and indeed there's a FIXME that says it should use an MVar. I didn't look very hard, though. peace, isaac ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
Re: concurrency (was Re: Priorities)
On Fri, 3 Feb 2006, Ross Paterson wrote: As another example, Ben Rudiak-Gould recently pointed out that the inclusion of stToIO breaks threaded state reasoning for ST, e.g. readSTRef won't necessarily get what your last writeSTRef wrote (because the region might be RealWorld, with other threads modifying it). You can still reason about something of type ST s a, it's just with the proviso that the reasoning is only correct when it is (perhaps indirectly) invoked by runST. Cheers, Ganesh ___ Haskell-prime mailing list Haskell-prime@haskell.org http://haskell.org/mailman/listinfo/haskell-prime
