I took a brief gander at the OpenLava source code, and a couple of things jump out. First, OpenLava is a batch scheduler and only supports batch execution - there is no interactive command for "run this job". So you would have to "bsub" mpirun regardless.
Once you submit the job, mpirun can certainly read the local allocation via the environment. However, we cannot use the OpenLava internal functions to launch the daemons or processes as the code is GPL2, and thus has a viral incompatible license. Ordinarily, we get around that by just executing the interactive job execution command, but OpenLava doesn't have one. So we'd have no other choice but to use ssh to launch the daemons on the remote nodes. This is exactly what the provided openmpi wrapper script that comes with OpenLava already does. Bottom line: I don't see a way to do any deeper integration minus the interactive execution command. If OpenLava had a way of getting an allocation and then interactively running jobs, we could support what you requested. This doesn't seem to be what they are intending, unless I'm missing something (the documentation is rather incomplete). Ralph On Tue, Nov 18, 2014 at 6:20 AM, Marc Höppner <marc.hoepp...@bils.se> wrote: > Hi, > > sure, no problem. And about the C Api, I really don’t know more than what > I was told in the google group post I referred to (i.e. the API is > essentially identical to LSF 4-6, which should be on the web). > > The output of env can be found here: > https://dl.dropboxusercontent.com/u/1918141/env.txt > > /M > > Marc P. Hoeppner, PhD > Team Leader > BILS Genome Annotation Platform > Department for Medical Biochemistry and Microbiology > Uppsala University, Sweden > marc.hoepp...@bils.se > > On 18 Nov 2014, at 15:14, Ralph Castain <r...@open-mpi.org> wrote: > > If you could just run a single copy of "env" and send the output along, > that would help a lot. I'm not interested in the usual path etc, but would > like to see the envars that OpenLava is setting. > > Thanks > Ralph > > > On Tue, Nov 18, 2014 at 2:19 AM, Gilles Gouaillardet < > gilles.gouaillar...@iferc.org> wrote: > >> Marc, >> >> the reply you pointed is a bit confusing to me : >> >> "There is a native C API which can submit/start/stop/kill/re queue jobs" >> this is not what i am looking for :-( >> >> "you need to make an appropriate call to openlava to start a remote >> process" >> this is what i am interested in :-) >> could you be more specific (e.g. point me to the functions, since the >> OpenLava doc is pretty minimal ...) >> >> the goal here is to spawn the orted daemons as part of the parallel job, >> so these daemons are accounted within the parallel job. >> /* if we use an API that simply spawns orted, but the orted is not >> related whatsoever to the parallel job, >> then we can simply use ssh */ >> >> Cheers, >> >> Gilles >> >> >> On 2014/11/18 18:24, Marc Höppner wrote: >> >> Hi Gilles, >> >> thanks for the prompt reply. Yes, as far as I know there is a C API to >> interact with jobs etc. Some mentioning here: >> https://groups.google.com/forum/#!topic/openlava-users/w74cRUe9Y9E >> <https://groups.google.com/forum/#!topic/openlava-users/w74cRUe9Y9E> >> <https://groups.google.com/forum/#!topic/openlava-users/w74cRUe9Y9E> >> >> >> /Marc >> >> Marc P. Hoeppner, PhD >> Team Leader >> BILS Genome Annotation Platform >> Department for Medical Biochemistry and Microbiology >> Uppsala University, swedenmarc.hoepp...@bils.se >> >> On 18 Nov 2014, at 08:40, Gilles Gouaillardet >> <gilles.gouaillar...@iferc.org> <gilles.gouaillar...@iferc.org> wrote: >> >> Hi Marc, >> >> OpenLava is based on a pretty old version of LSF (4.x if i remember >> correctly) >> and i do not think LSF had support for parallel jobs tight integration >> at that time. >> >> my understanding is that basically, there is two kind of direct >> integration : >> - mpirun launch: mpirun spawns orted via the API provided by the batch >> manager >> - direct launch: the mpi tasks are launched directly from the >> script/command line and no mpirun/orted is involved >> at that time, it works with SLURM and possibly other PMI capable batch >> manager >> >> i think OpenLava simply gets a list of hosts from the environment, build >> a machinefile, pass it to mpirun that spawns orted with ssh, so this is >> really loose integration. >> >> OpenMPI is based on plugins, so as long as the queing system provides an >> API to start/stop/kill tasks, mpirun launch should not >> be a huge effort. >> >> Are you aware of such an API provided by OpenLava ? >> >> Cheers, >> >> Gilles >> >> On 2014/11/18 16:31, Marc Höppner wrote: >> >> Hi list, >> >> I have recently started to wonder how hard it would be to add support for >> queuing systems to the tight integration function of OpenMPI (unfortunately, >> I am not a developer myself). Specifically, we are working with OpenLava >> (www.openlava.org), which is based on an early version of Lava/LSF and open >> source. It’s proven quite useful in environments where some level of LSF >> compatibility is needed, but without actually paying for a (rather pricey) >> LSF license. >> >> Given that openLava shares quite a bit of DNA with LSF, I was wondering how >> hard it would be to add OL tight integration support to OpenMPI. Currently, >> OL enables OpenMPI jobs through a wrapper script, but that’s obviously not >> ideal and doesn’t work for some programs that have MPI support built-in (and >> thus expect to be able to just execute mpirun). >> >> Any thoughts on this would be greatly appreciated! >> >> Regards, >> >> Marc >> >> Marc P. Hoeppner, PhD >> Team Leader >> BILS Genome Annotation Platform >> Department for Medical Biochemistry and Microbiology >> Uppsala University, swedenmarc.hoepp...@bils.se >> >> _______________________________________________ >> devel mailing listde...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> Link to this post: >> http://www.open-mpi.org/community/lists/devel/2014/11/16312.php >> >> _______________________________________________ >> devel mailing listde...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> Link to this post: >> http://www.open-mpi.org/community/lists/devel/2014/11/16313.php >> >> >> >> _______________________________________________ >> devel mailing listde...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> >> Link to this post: >> http://www.open-mpi.org/community/lists/devel/2014/11/16314.php >> >> >> >> _______________________________________________ >> devel mailing list >> de...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel >> Link to this post: >> http://www.open-mpi.org/community/lists/devel/2014/11/16315.php >> > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/11/16316.php > > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/11/16317.php >
