> Am 26.07.2017 um 15:09 schrieb r...@open-mpi.org: > > mpirun doesn’t get access to that requirement, nor does it need to do so. SGE > will use the requirement when determining the nodes to allocate.
m_mem_free appears to come from Univa GE and is not part of the open source versions. So I can't comment on this for sure, but it seems to set the memory also in cgroups. -- Reuti > mpirun just uses the nodes that SGE provides. > > What your cmd line does is restrict the entire operation on each node (daemon > + 8 procs) to 40GB of memory. OMPI does not support per-process restrictions > other than binding to cpus. > > >> On Jul 26, 2017, at 6:03 AM, Kulshrestha, Vipul >> <vipul_kulshres...@mentor.com> wrote: >> >> Thanks for a quick response. >> >> I will try building OMPI as suggested. >> >> On the integration with unsupported distribution systems, we cannot use >> script based approach, because often these machines don’t have ssh >> permission in customer environment. I will explore the path of writing orte >> component. At this stage, I don’t understand the effort for the same. >> >> I guess my question 2 was not understood correctly. I used the below command >> as an example for SGE and want to understand the expected behavior for such >> a command. With the below command, I expect to have 8 copies of a.out >> launched with each copy having access to 40GB of memory. Is that correct? I >> am doubtful, because I don’t understand how mpirun gets access to >> information about RAM requirement. >> >> qsub –pe orte 8 –b y –V –l m_mem_free=40G –cwd mpirun –np 8 a.out >> >> >> Regards, >> Vipul >> >> >> >> From: users [mailto:users-boun...@lists.open-mpi.org] On Behalf Of >> r...@open-mpi.org >> Sent: Tuesday, July 25, 2017 8:16 PM >> To: Open MPI Users <users@lists.open-mpi.org> >> Subject: Re: [OMPI users] Questions about integration with resource >> distribution systems >> >> >> On Jul 25, 2017, at 3:48 PM, Kulshrestha, Vipul >> <vipul_kulshres...@mentor.com> wrote: >> >> I have several questions about integration of openmpi with resource queuing >> systems. >> >> 1. >> I understand that openmpi supports integration with various resource >> distribution systems such as SGE, LSF, torque etc. >> >> I need to build an openmpi application that can interact with variety of >> different resource distribution systems, since different customers have >> different systems. Based on my research, it seems that I need to build a >> different openmpi installation to work, e.g. create an installation of >> opempi with grid and create a different installation of openmpi with LSF. Is >> there a way to build a generic installation of openmpi that can be used with >> more than 1 distribution system by using some generic mechanism? >> >> Just to be clear: your application doesn’t depend on the environment in any >> way. Only mpirun does - so if you are distributing an _application_, then >> your question is irrelevant. >> >> If you are distributing OMPI itself, and therefore mpirun, then you can >> build the various components if you first install the headers for that >> environment on your system. It means that you need one machine where all >> those resource managers at least have their headers installed on it. Then >> configure OMPI --with-xxx pointing to each of the RM’s headers so all the >> components get built. When the binary hits your customer’s machine, only >> those components that have active libraries present will execute. >> >> >> 2. >> For integration with LSF/grid, how would I specify the memory (RAM) >> requirement (or some other parameter) to bsub/qsub, when launching mpirun >> command? Will something like below work to ensure that each of the 8 copies >> of a.out have 40 GB memory reserved for them by grid engine? >> >> qsub –pe orte 8 –b y –V –l m_mem_free=40G –cwd mpirun –np 8 a.out >> >> You’ll have to provide something that is environment dependent, I’m afraid - >> there is no standard out there. >> >> >> >> 3. >> Some of our customers use custom distribution engine (some >> non-industry-standard distribution engine). How can I integrate my openmpi >> application with such system? I would think that it should be possible to do >> that if openmpi launched/managed interaction with the distribution engine >> using some kind of generic mechanism (say, use a configurable command to >> launch, monitor, kill a job and then allow specification of a plugin define >> these operations with commands specific to the distribution engine being in >> use). Does such integration exist in openmpi? >> >> Easiest solution is to write a script that reads the allocation and dumps it >> into a file, and then provide that file as your hostfile on the mpirun cmd >> line (or in the environment). We will then use ssh to perform the launch. >> Otherwise, you’ll need to write at least an orte/mca/ras component to get >> the allocation, and possibly an orte/mca/plm component if you want to use >> the native launch mechanism in place of ssh. >> >> >> >> >> Thanks, >> Vipul >> >> >> _______________________________________________ >> users mailing list >> users@lists.open-mpi.org >> https://rfd.newmexicoconsortium.org/mailman/listinfo/users >> >> _______________________________________________ >> users mailing list >> users@lists.open-mpi.org >> https://rfd.newmexicoconsortium.org/mailman/listinfo/users > > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://rfd.newmexicoconsortium.org/mailman/listinfo/users _______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
