On Fri, 2011-08-05 at 23:51 -0400, dsimcha wrote: [ . . . ] > 1. Is anyone besides me interested in this?
MPI may be ancient, it may be a bit daft in terms of its treatment of marshalling, unmarshalling and serializing, it may be only a Fortran and C thing bolted into C++ (quite well) but it is the de facto standard for HPC. OK so HPC is about 10% of world-wide computing, probably less than that of spend despite the enormous per installation price, but it is about 90% of political marketing. Any short term parallelism strategy must include MPI -- and work with OpenMPI and MPICH2. So I don't think it is a matter of just interest for D, I would say that if D is to stand with C++, C and Fortran then there has to be an MPI API. Even though MPI should be banned going forward. > 2. Is anyone already working on something similar. > > 3. Would this be Phobos material even though it would depend on MPI, or > would it better be kept as a 3rd party library? Given that it requires a transitive dependency then either Phobos goes forward with optional dependencies or the MPI API is a separate thing. Given my personal opinion that actor model, dataflow model, agents, etc. should be the application level concurrency and parallelism model, I would be quite happy with an MPI API not being in Phobos. Keep Phobos for that which every D installation will need. MPI is a niche market in that respect. Optional dependencies sort of work but are sort of a real pain in the Java/Maven milieu. > 4. std.concurrency in its current incarnation doesn't allow objects > with mutable indirection to be passed as messages. This makes sense > when passing messages between threads in the same address space. > However, for passing between MPI processes, the object is going to be > copied anyhow. Should the restriction be kept (for consistency) or > removed (because it doesn't serve much of a purpose in the MPI context)? At the root of this issue is local thread-based parallelism in a shared memory context, vs cluster parallelism. MPI is a cluster solution -- even though it can be used in multicore shared memory situation. The point about enforced copying vs. potential sharing is core to this obviously. This has to be handled with absolute top notch performance in mind. It is arguably a situation where programming language semantics and purity have to be sacrificed at the altar of performance. There are already far too many MPI applications that are written with far too much comms code in the application simply to ensure performance -- because the MPI infrastructure cannot be trusted to do things fast enough if you use anything other than the bottom most layer. > 5. For passing complex object graphs, serialization would obviously be > necessary. What's the current state of the art in serialization in D? > I want something that's efficient and general first and foremost. I > really don't care about human readability or standards compliance (in > other words, no XML or JSON or anything like that). Again performance is everything, so nothing must get in the way of having something that cannot be made faster. The main problem here is going to be that when anything gets released performance will be the only yardstick by which things are measured. Simplicity of code, ease of evolution of code, all the things professional developers value, will go out of the window. It's HPC after all :-) I still think D needs a dataflow, CSP and data parallelism strategy, cf. Go, GPars, Akka, even Haskell. Having actors is good, but having only actors is not good, cf. Scala and Akka. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc
Description: This is a digitally signed message part
