On Tue, Dec 22, 2009 at 6:40 AM, Linas Vepstas <linasveps...@gmail.com> wrote: > However, if you are interested in merely using the system > to do your "real" work, then writing message-passing code > is an utter waste of time -- its difficult, time-consuming, error > prone, hard to balance and optimize & tune, works well only > for "embarrasingly parallel" code, etc. Even the evil > slow-down of NUMA is often better than trying to > performance-tune a message-passing system.
Message passing doesn't work well only for embarrassingly parallel code. For instance, you can implement the aforementioned parallel quicksort rather easily, but it's true that message passing is low-level. The really bad thing about MPI is that it assumes some C-like environment. C has the worst semantics ever, so programs that require/encourage such a style of writing are inherently, well, bad =) And yes, MPI's difficult to debug. What message passing really is, it is the perfect match to a distributed memory architecture. Since, as you suggest, multicore chips have more or less a shared memory architecture, message passing is indeed not a good match. However, let's not forget about the new GPU architectures, which are sort of hybrid. The newer GPUs will have more exotic on-chip interconnection networks. > Let me put it this way: suggesting that programmers can > write their own message-passing system is kind of like > telling them that they can write their own garbage-collection > system, or design their own closures, or they can go > create their own type system. Of course they can ... and > if they wanted to do that, they would be programming in > C or assembly, and would probably be designing new > languages. Cause by the time you get done with message > passing, you've created a significant and rich programming > system that resembles a poorly-designed language... been > there, done that. For a functional language, am I right in expecting a high-level and clean interface for explicit parallelism? I suppose a "spawn" directive would not be very hard to implement. Or a parallel let block. I don't know what kind of direction the caml researchers have in mind (except for some INRIA projects I've read about) but I suppose it can be done (closures could be a nice interface, I think). Message Passing/Distributed Memory can also be accommodated I suppose. For distributed memory, you just need an addressing scheme to denote the processor number. You could allocate with a parallel new that takes a processor number argument for instance. For message passing, you need one-to-one, collective, and one-sided communications. On top of an imperative but failsafe and debuggable interface, you could provide neat functional blocks. For instance, you could present the user with a shuffle/map/reduce interface like in Google's grid computing platform. I would also like to have some computing-abstraction (like Monads, but more flexible) so I can easily build networks of sequential algorithms (kind of like Communicating Sequential Processses). It would also be nice to have functors that implement commonly occurring parallel programming patterns (such as master-slave dynamic load balancing or pipelining). Such things are difficult to manage in a low-level language like C, but they would be a piece of cake in ocaml. Or is fate sending that my way? Oh, not again! :) OcamlP3l looks pretty cool. Parallel combinators? Definitely what I'm talking about, as usual the future is here with ocaml ;) http://ocamlp3l.inria.fr/eng.htm Best, -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct _______________________________________________ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs