Re: AIO and threads - my prejudices
On Tue, Jan 09, 2001 at 11:44:55PM +, Alan Burlison wrote: > The downside is that we would restrict perl6 to only those platform that > supported threads. I'm not sure how much of a restriction this would > turn out to be in practice - how many current perl platforms don't have > threads? As mentioned previously Acorn's RISC OS doesn't However, I've just had a talker session and got a tutorial from the guy who wrote their Java implementation on how to write a threading library that would run command line programs. (specifically the tricky bits of how to swap thread context, and how to do this while being pre-empted inside the desktop without crashing everyone else) We didn't get to "how to do this and also be able to thread while displaying windows of your own in the desktop" because he can't remember offhand how he was going to do this for the general case) (java does do it, but as it's all going through Java's windowing library, the implementation know where the windowing system calls are going to be) Upshot is that threading can be written if need be, so that's one less (admittedly now obscure) platform that isn't excluded by threading. Nicholas Clark
Re: AIO and threads - my prejudices
Alan Burlison wrote: > Hmm. I've been half-following the async IO and signals thread in > perl6-internals. The first thing I would say is that if you think there > are portability problems with threads and signals, wait until you get to > play with cross-platform AIO. General received wisdom seems to be that > using multiple threads and synchronous IO is a much easier thing to get > working than trying to use the various difficult-to-use AIO APIs. > > ... > > The downside is that we would restrict perl6 to only those platform that > supported threads. I'm not sure how much of a restriction this would > turn out to be in practice - how many current perl platforms don't have > threads? > > As for AIO - my guess is that faking it up with threads is a much better > bet. After all, what proportion of apps are MT vs AIO, and which is > most likely to be available, well tested and well supported? > > Alan Burlison > Well, I guess this is the time to see all the options we have, so we should do it now. As Alan said and I agree, there are portability issues with threads, AIO and signals. But I believe many of us don't know what they are and also don't know how to deal with them. Well, I think now is the time to examine it with real attention so we can possibly go into a direction to a solution based on real facts, not only on conjectures. As it seems, we have many options for implementing perl's event loop: 1) use async i/o and implement threads internally, switching the context of the interpreters. 2) use threads and no async i/o, blocking the event loop when a blocking syscall is made (how would signals be handled?). 3) use async i/o and system threads where available. 4) not using async i/o or threads. (i don't know) what can we do on platforms where neither threads nor async i/o are not available? 5) having more than one implementations, for systems with different degrees of support. Combining chooses we make on threads, async i/o, signals, threads<->event loops, and other of those relevant things, we have several options to consider, that can even affect the language (for example, will perl-level threads be offered on systems that don't support them but support async i/o?). I believe we should not talk anymore about, for example, using aio or not aio, or using threads or not using threads. I think we should talk about whole scenarios, for example, using aio but not using system threads, there is one event loop that does context switching, ... For this to be possible, we have to list and describe in details what are our options. Some of them I listed above, but I can't give enough details, I'm no expert on this. Having a list of our possibilities, we can effectively talk about advantages and disadvantages (also efficiency) of them. Other thing to be considered is portability (again, portability of a scenario, not of aio or threads isolated). I don't really know (or have access) to many platforms, I think the people on p5p are the ones that can help better on this discussion. I think we should take facts about the implementation of aio, threads, signals, ... on these platforms. I think our target is to make perl6 run in _ALL_ platforms that run perl5 today, but I actually can be wrong: maybe, given the options, it's decided that platforms that don't support aio/threads won't have perl6 (???). As for portability, I think we should try to write down a table with all the platforms today supported by perl5, with availability (and possible implementation hints) of aio, threads, (signals?) in all those platforms. I suggest we work on making a test suite for these features, testing event loops and operations that can afterwards be used as a base for perl's kernel. Efficiency benchmarks can be taken here too. I think those results are the ones that matter for making a decision about which scenario will be used. I guess that choice is for Larry, but we should give him enough facts about the scenarios so that the choice is made right and we don't have to redesign because of portability/efficiency problems. Branden.
AIO and threads - my prejudices
Copied from p5p as it seemed kinda relevant. Dan Sugalski wrote: > >Roll on perl6... > > Well, besides "Just don't *do* that," any thoughts on how to handle this > properly in p6? Hmm. I've been half-following the async IO and signals thread in perl6-internals. The first thing I would say is that if you think there are portability problems with threads and signals, wait until you get to play with cross-platform AIO. General received wisdom seems to be that using multiple threads and synchronous IO is a much easier thing to get working than trying to use the various difficult-to-use AIO APIs. That leads nicely onto the next point - signals. Again, the received wisdom here is that if you are going to use threads and signals together, you should have a dedicated signal handling thread which does a sigwait and sets a flag when a signal occurs. The flag would be polled by the main loop and the signal handler called when it was safe to do so. Now I can see that at this point all you speed freaks want jump on me and tell me how horribly inefficient this additional test would be. I have only one thing to say to you - get real. One conditional (even if protected by a mutex) is neither here nor there when compared to the number of instructions taken to implement a typical perl op, and you wouldn't do it for every op dispatched anyway. A sensible thing might to be to get the compiler to emit statement boundary markers into its output, and use these as points at which you check for and dispatch signals. For example, on my platform (sparc), an empty for loop takes about 10 cycles per iteration, a function call to setjmp takes about 10 cycles and a call to sigsetjmp takes more than 5300 cycles, due largely to the fact that it requires a syscall and is therefore also an invitation to the OS to reschedule your process. If you want to worry about something, worry about that 5300 cycles first. Then worry even more that setjmp isn't thread safe either. The upshot of all this is I think perl6 should be mandatorally threaded, with a mixture of internal threads (signal handler, async IO handler, garbage collector, whatever), and application threads. Each application thread would be an entire instance of the interpreter - none of that crazy mutex madness that doomed the attempt to thread perl5. Interpreter threads would touch at well-defined points only - the aim being to avoid mutexes as far as possible, rather than to infect everything with them. In the degenerate case (existing unthreaded scripts) there would be only one interpreter instance and therefore only one application thread. This has lots of advantages - no changes in behaviour when threads are used as perl is always threaded, well-defined semantics of how multiple interpreters coexist and cooperate, system housekeeping can be modularised and done behind the scenes in dedicated threads etc etc. The downside is that we would restrict perl6 to only those platform that supported threads. I'm not sure how much of a restriction this would turn out to be in practice - how many current perl platforms don't have threads? As for AIO - my guess is that faking it up with threads is a much better bet. After all, what proportion of apps are MT vs AIO, and which is most likely to be available, well tested and well supported? Alan Burlison
