[
https://issues.apache.org/jira/browse/MAPREDUCE-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093838#comment-13093838
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2911:
----------------------------------------------------
This is a great addition! +1 for the simple design.
I guess the client spins till all the containers are allocated (Oh it's HOD all
over again :) )
Like MR, the client could continuously ping the AM via RPC to know about the
status of container-allocations. Presence/absence of HDFS file sure is a simple
beginning.
bq. Also, if any of the container (running orte), exits abnormally, entire
virtual MPI cluster is terminated. (This limitation will be removed in the next
version.)
How does the mpi client get to know about this? It'd be great if mpi-run
automatically detects this.
Security can be postponed for the first-cut.
For now, +1 to hadoop-openmpi module under hadoop-mapreduce-project. Once we
have this, we will have a platform(yarn) and two frameworks(MR and MPI) using
it. Then we can move out yarn out of mapreduce-project, mapreduce and MPI can
move into a frameworks aggregation module (the app-store, gawd, I can't believe
I just said that :) )
> Hamster: Hadoop And Mpi on the same cluSTER
> -------------------------------------------
>
> Key: MAPREDUCE-2911
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: All Unix-Environments
> Reporter: Milind Bhandarkar
> Assignee: Milind Bhandarkar
> Fix For: 0.23.0
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> MPI is commonly used for many machine-learning applications. OpenMPI
> (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the
> past, running MPI application on a Hadoop cluster was achieved using Hadoop
> Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was
> kludgy. After the resource-manager separation from JobTracker in Hadoop, we
> have all the tools needed to make MPI a first-class citizen on a Hadoop
> cluster. I am currently working on the patch to make MPI an
> application-master. Initial version of this patch will be available soon
> (hopefully before September 10.) This jira will track the development of
> Hamster: The application master for MPI.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
