Integrating MPI with map-reduce is currently difficult and/or very ugly,
however. Not impossible and there are hackish ways to do the job, but
they are hacks.
There is an project out of Sandia National Lab that puts MR and MPI
together in a library if you're interested --
On Sat, Apr 30, 2011 at 12:18 AM, elton sky eltonsky9...@gmail.com wrote:
I got 2 questions:
1. I am wondering how hadoop MR performs when it runs compute intensive
applications, e.g. Monte carlo method compute PI. There's a example in
0.21,
QuasiMonteCarlo, but that example doesn't use
Ted,
MPI supports node-to-node communications in ways that map-reduce does not,
however, which requires that you iterate map-reduce steps for many
algorithms. With Hadoop's current implementation, this is horrendously
slow (minimum 20-30 seconds per iteration).
Sometimes you can avoid this
