FWIW: the “better” solution is to move Hadoop to an HPC-like RM such as Slurm.
We did this as Pivotal as well as at Intel, but in both cases business moves at
the very end of the project (Greenplum becoming Pivotal, and Intel moving its
Hadoop work into Cloudera) blocked its release. Frustrating
Thanks this is good feedback.
I was worried with the dynamic nature of Yarn containers that it would be hard
to coordinate wire up, and you have confirmed that.
Thanks
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
bro...@umich.edu
(734)936-1985
> On Oct 27,
> On Oct 26, 2014, at 9:56 PM, Brock Palen wrote:
>
> We are starting to look at supporting MPI on our Hadoop/Spark YARN based
> cluster.
You poor soul…
> I found a bunch of referneces to Hamster, but what I don't find is if it was
> ever merged into regular OpenMPI, and if so is it just an
