mpirun doesn’t get access to that requirement, nor does it need to do so. SGE will use the requirement when determining the nodes to allocate. mpirun just uses the nodes that SGE provides.
What your cmd line does is restrict the entire operation on each node (daemon + 8 procs) to 40GB of memory. OMPI does not support per-process restrictions other than binding to cpus. > On Jul 26, 2017, at 6:03 AM, Kulshrestha, Vipul > <vipul_kulshres...@mentor.com> wrote: > > Thanks for a quick response. > > I will try building OMPI as suggested. > > On the integration with unsupported distribution systems, we cannot use > script based approach, because often these machines don’t have ssh permission > in customer environment. I will explore the path of writing orte component. > At this stage, I don’t understand the effort for the same. > > I guess my question 2 was not understood correctly. I used the below command > as an example for SGE and want to understand the expected behavior for such a > command. With the below command, I expect to have 8 copies of a.out launched > with each copy having access to 40GB of memory. Is that correct? I am > doubtful, because I don’t understand how mpirun gets access to information > about RAM requirement. > > qsub –pe orte 8 –b y –V –l m_mem_free=40G –cwd mpirun –np 8 a.out > > > Regards, > Vipul > > > <> > From: users [mailto:users-boun...@lists.open-mpi.org] On Behalf Of > r...@open-mpi.org > Sent: Tuesday, July 25, 2017 8:16 PM > To: Open MPI Users <users@lists.open-mpi.org> > Subject: Re: [OMPI users] Questions about integration with resource > distribution systems > > > On Jul 25, 2017, at 3:48 PM, Kulshrestha, Vipul <vipul_kulshres...@mentor.com > <mailto:vipul_kulshres...@mentor.com>> wrote: > > I have several questions about integration of openmpi with resource queuing > systems. > > 1. > I understand that openmpi supports integration with various resource > distribution systems such as SGE, LSF, torque etc. > > I need to build an openmpi application that can interact with variety of > different resource distribution systems, since different customers have > different systems. Based on my research, it seems that I need to build a > different openmpi installation to work, e.g. create an installation of opempi > with grid and create a different installation of openmpi with LSF. Is there a > way to build a generic installation of openmpi that can be used with more > than 1 distribution system by using some generic mechanism? > > Just to be clear: your application doesn’t depend on the environment in any > way. Only mpirun does - so if you are distributing an _application_, then > your question is irrelevant. > > If you are distributing OMPI itself, and therefore mpirun, then you can build > the various components if you first install the headers for that environment > on your system. It means that you need one machine where all those resource > managers at least have their headers installed on it. Then configure OMPI > --with-xxx pointing to each of the RM’s headers so all the components get > built. When the binary hits your customer’s machine, only those components > that have active libraries present will execute. > > > 2. > For integration with LSF/grid, how would I specify the memory (RAM) > requirement (or some other parameter) to bsub/qsub, when launching mpirun > command? Will something like below work to ensure that each of the 8 copies > of a.out have 40 GB memory reserved for them by grid engine? > > qsub –pe orte 8 –b y –V –l m_mem_free=40G –cwd mpirun –np 8 a.out > > You’ll have to provide something that is environment dependent, I’m afraid - > there is no standard out there. > > > > 3. > Some of our customers use custom distribution engine (some > non-industry-standard distribution engine). How can I integrate my openmpi > application with such system? I would think that it should be possible to do > that if openmpi launched/managed interaction with the distribution engine > using some kind of generic mechanism (say, use a configurable command to > launch, monitor, kill a job and then allow specification of a plugin define > these operations with commands specific to the distribution engine being in > use). Does such integration exist in openmpi? > > Easiest solution is to write a script that reads the allocation and dumps it > into a file, and then provide that file as your hostfile on the mpirun cmd > line (or in the environment). We will then use ssh to perform the launch. > Otherwise, you’ll need to write at least an orte/mca/ras component to get the > allocation, and possibly an orte/mca/plm component if you want to use the > native launch mechanism in place of ssh. > > > > > Thanks, > Vipul > > > _______________________________________________ > users mailing list > users@lists.open-mpi.org <mailto:users@lists.open-mpi.org> > https://rfd.newmexicoconsortium.org/mailman/listinfo/users > <https://rfd.newmexicoconsortium.org/mailman/listinfo/users> > > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://rfd.newmexicoconsortium.org/mailman/listinfo/users
_______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
