I heard about D recently from a thread in the Gentoo Forums. My question is, if I'm mostly interested in shared memory high performance computing, do the radical new D 2 concurrency features help me at all? What I have in mind is doing automatic map/reduce style of programming.
Sounds like http://cis.jhu.edu/~dsimcha/d/phobos/std_parallelism.html will be your friend. It's currently in the review stage, see thread "David Simcha's std.parallelism".
I'm going to buy a new hardware providing the AVX instruction set, possibly a two socket system. This should improve loop vectorization a lot. How should it all work is via the "native" foreach and Phobos 2 library features. Is this possible? Does Phobos 2 also use the three level cache architecture in Sandy Bridge efficiently? I was considering D because I don't want to mess with low level assembly and use a high level modern language instead. C++ is also getting there, in GCC 4.6 the loop vectorization is already good.
Well array (or "vector") operations are currently optimized to leverage SSE - but this is no compiler optimization, it's hand-tuned code. Apart from that there's GDC (using the gcc backend) and LDC (using LLVM as the backend) which potentially optimize a lot.
