[OMPI users] launching the mpi runtime
Hi, One typically uses mpirun to launch a set of mpi processes. Is there some programmatical interface to launching the runtime and having the process that launched the runtime becoming part of the list of mpi processes, Regards,
Re: [OMPI users] launching the mpi runtime
you could spawn more processes from currently running processes. On Thu, Nov 18, 2010 at 3:05 AM, Hicham Mouline wrote: > Hi, > > One typically uses mpirun to launch a set of mpi processes. > > Is there some programmatical interface to launching the runtime and having > the process that launched the runtime becoming part of the list of mpi > processes, > > > > Regards, > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > -- David Zhang University of California, San Diego
Re: [OMPI users] launching the mpi runtime
Other than MPI_COMM_SPAWN[_MULTIPLE], we don't expose the underlying run-time to MPI applications. There is a whole programatic interface for the layer under MPI (the Open MPI Runtime Environment -- ORTE), though. We don't advise mixing ORTE calls in MPI applications, but it is certainly feasible to use ORTE for non-MPI things (some of the OMPI community member organizations do so). What are you trying to do? On Nov 18, 2010, at 11:37 AM, David Zhang wrote: > you could spawn more processes from currently running processes. > > On Thu, Nov 18, 2010 at 3:05 AM, Hicham Mouline wrote: > Hi, > > One typically uses mpirun to launch a set of mpi processes. > > Is there some programmatical interface to launching the runtime and having > the process that launched the runtime becoming part of the list of mpi > processes, > > > Regards, > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > -- > David Zhang > University of California, San Diego > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] launching the mpi runtime
i have now a application with a gui. The gui launches the calculations in-process serially. No MPI involved. Fine. The objective is to parallelize. I want to keep the GUI(windows) as the control to start calcs and display results. The GUI could be the master process of the mpi processes. That's bad because the executable image has deps on the GUI library and there's no need for all the mpi processes (the same executable) to have anything to do with the display. besides, i have a win box and a couple of linux boxes, and openmpi cannot mix both in the same group of mpi processes. therefore, I guess I need to separate the GUI binary from the mpi-processes binary and have the GUI process talk to the "master" mpi process (on linux) for calc requests. I was hoping I wouldn't have to write a custom code to do that. > -Original Message- > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On > Behalf Of Jeff Squyres > Sent: 22 November 2010 15:53 > To: Open MPI Users > Subject: Re: [OMPI users] launching the mpi runtime > > Other than MPI_COMM_SPAWN[_MULTIPLE], we don't expose the underlying > run-time to MPI applications. > > There is a whole programatic interface for the layer under MPI (the > Open MPI Runtime Environment -- ORTE), though. We don't advise mixing > ORTE calls in MPI applications, but it is certainly feasible to use > ORTE for non-MPI things (some of the OMPI community member > organizations do so). > > What are you trying to do? > > > On Nov 18, 2010, at 11:37 AM, David Zhang wrote: > > > you could spawn more processes from currently running processes. > > > > On Thu, Nov 18, 2010 at 3:05 AM, Hicham Mouline > wrote: > > Hi, > > > > One typically uses mpirun to launch a set of mpi processes. > > > > Is there some programmatical interface to launching the runtime and > having the process that launched the runtime becoming part of the list > of mpi processes, > > > > > > Regards, > > > > > > ___ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > > > > > -- > > David Zhang > > University of California, San Diego > > ___ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] launching the mpi runtime
On Nov 22, 2010, at 11:52 AM, Hicham Mouline wrote: > i have now a application with a gui. The gui launches the calculations > in-process serially. > No MPI involved. Fine. The objective is to parallelize. Gotcha. > I want to keep the GUI(windows) as the control to start calcs and display > results. > > The GUI could be the master process of the mpi processes. > That's bad because the executable image has deps on the GUI library and > there's no need for all the mpi processes (the same executable) to have > anything to do with the display. Sounds reasonable. You likely want to have (at least) 2 executables, then: the GUI and the compute worker. > besides, i have a win box and a couple of linux boxes, and openmpi cannot > mix both in the same group of mpi processes. Sadly true. There has been (very) little demand for this, so we haven't spent much (any) time on making it work. > therefore, I guess I need to separate the GUI binary from the mpi-processes > binary and have the GUI process talk to the "master" mpi process (on linux) > for calc requests. > > I was hoping I wouldn't have to write a custom code to do that. MPI doesn't necessarily mean SPMD -- you can certainly have the GUI call MPI_INIT and then call MPI_COMM_SPAWN to launch a different executable to do the compute working stuff. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] launching the mpi runtime
>MPI doesn't necessarily mean SPMD -- you can certainly have the GUI call >MPI_INIT and then call MPI_COMM_SPAWN to launch a different >executable to do >the compute working stuff. >-- >Jeff Squyres >jsquyres_at_[hidden] This is confusing to me. If the GUI does that, will the GUI process (running on windows) be able to spawn mpi processes on separate linux boxes and "MPI talk" to them using the intercommunicator returned? rds,
Re: [OMPI users] launching the mpi runtime
>> therefore, I guess I need to separate the GUI binary from the mpi-processes >> binary and have the GUI process talk to the "master" mpi process (on linux) >> for calc requests. >> >> I was hoping I wouldn't have to write a custom code to do that. >MPI doesn't necessarily mean SPMD -- you can certainly have the GUI call MPI_INIT and then call MPI_COMM_SPAWN to launch a >different executable to do the compute working stuff. Hi Jeff, I couldn't quite understand this last statement. If the GUI's on windows, and the compute processes are on linux for instance, no MPI communication is possible right. That means, I do need to write custom code to communicate between the GUI process (win) and the mpi master (linux) regards,
Re: [OMPI users] launching the mpi runtime
On Nov 23, 2010, at 8:34 AM, Hicham Mouline wrote: >> MPI doesn't necessarily mean SPMD -- you can certainly have the GUI call >> MPI_INIT and then call MPI_COMM_SPAWN to launch a different >executable to >> do the compute working stuff. > > This is confusing to me. > If the GUI does that, will the GUI process (running on windows) be able to > spawn mpi processes on separate linux boxes and "MPI talk" to them using the > intercommunicator returned? Sorry for the delayed reply -- SC10 and the US holiday got in the way. All I mean here is that your MPI job doesn't have to consist of a single executable. You can have a GUI front-end application that MPI_COMM_SPAWN's non-GUI worker processes on the back-end nodes. The point of MPI_COMM_SPAWN is that you can spawn some other MPI processes and get an intercommuicator back that you can use for normal MPI communications. To be clear, if your GUI calls MPI_COMM_SPAWN to launch non-GUI "worker" executable on the compute nodes, then you can use the intercommunicator that is returned via MPI_COMM_SPAWN to MPI_SEND / MPI_RECV / etc. with the new processes. The new processes will need to call MPI_COMM_GET_PARENT to get the intercommunicator that is shared with the parent process(es). Make sense? Read through the MPI spec section on "dynamic processes" for more details. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] launching the mpi runtime
On Nov 24, 2010, at 4:41 PM, Hicham Mouline wrote: >> Sadly true. There has been (very) little demand for this, so we >> haven't spent much (any) time on making it work. > > Is it the same for MacOSX? can't mix MacOsX and linux? > Can't mix Win and MacOSX? The issue is mainly Windows, I believe. I *believe* that the network transport that Windows uses is not compatible with any other. Shiqing -- can you confirm/deny? Hence, you can run OSX + Linux MPI jobs, although you need to be careful about exactly how you do it -- heterogeneity is not for the meek. You need to have different executables for your different OS's, for example. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI users] launching the mpi runtime
On Nov 29, 2010, at 2:29 PM, Hicham Mouline wrote: > If the GUI's on windows, and the compute processes are on linux for > instance, no MPI communication is possible right. > That means, I do need to write custom code to communicate between the GUI > process (win) and the mpi master (linux) With Open MPI, correct -- we don't currently support "MPI talk" between Windows and Linux. Sorry! :-( -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
