[OMPI devel] running Open MPI with different install paths

Folks, i am trying to run heterogeneous Open MPI. all my nodes use NFS everything is shared, so i need to manually specify x86_64 nodes must use /.../ompi-x86_64 and sparcv9 nodes must use /.../ompi-sparcv9 is there a simple way to achieve this ? Cheers, Gilles

Re: [OMPI devel] running Open MPI with different install paths

Hi Gilles What launch environment? We don't currently have a simple way of doing this outside of ensuring the paths on those nodes point to the correct default place (i.e., you can't use prefix). However, it might be possible to add such support if we knew which nodes were what type. Unfortunately

Re: [OMPI devel] running Open MPI with different install paths

Back in the days when I worked on heterogeneous machines like this, I had logic in my shell startup files to set paths properly. E.g. (pseudocode): - arch=`config.guess` switch $arch: case *x86_64-linux*) prefix_path=$HOME/x86_64-linux-stuff/bin prefix_ldpath=$HOME/x86_64-linux-stuff/lib

[OMPI devel] VERSION numbers for v1.8.5

I reviewed the v1.8 logs and I think that this is what the shared library version numbers should be. Essentially: most have minor code changes, meaning that they should get an revision bump (the "r" in c:r:a). And the rest should get also get a revision bump because mutex.h changed. This latt

[OMPI devel] v1.8.5 NEWS and README

The v1.8 branch NEWS, README, and VERSION files have been updated in preparation for the v1.8.5 release. Please double check them -- especially NEWS, particularly to ensure that we are giving credit to users who submitted bug reports, etc. Also, please double check that this is a current/corre

Re: [OMPI devel] running Open MPI with different install paths

Ralph, right now, I am using ssh one way to go is to extend the machine file syntax instead of user@host we could have user@host:port//prefix an other way would be to do this on the command line : mpirun --host host1 --prefix prefix1 a.out : -- host host2 --prefix prefix2 b.out an other really

Re: [OMPI devel] running Open MPI with different install paths

Will these kinds of things work in all launchers, or just ssh? I'm a little uncomfortable with going to extraordinary measures for a fairly uncommon scenario, especially when there are mechanisms that already exist that are designed for this kind of use case (i.e., logic in shell login/startup

Re: [OMPI devel] v1.8.5 NEWS and README

Hi Jeff Minor cray corrections below On Apr 17, 2015 6:57 AM, "Jeff Squyres (jsquyres)" wrote: > > The v1.8 branch NEWS, README, and VERSION files have been updated in preparation for the v1.8.5 release. Please double check them -- especially NEWS, particularly to ensure that we are giving cred

Re: [OMPI devel] v1.8.5 NEWS and README

Howard -- I notice that you have - Cray XE and XC on the master README. Which is correct for v1.8.5: XC or XK? > On Apr 17, 2015, at 10:02 AM, Howard Pritchard wrote: > > Hi Jeff > > Minor cray corrections below > > On Apr 17, 2015 6:57 AM, "Jeff Squyres (jsquyres)" wrote: > > > > The

[OMPI devel] mtt failures from last nite

HI Folks, I'm seeing build failures on both carver/pgi at nersc and on a cray internal machine with the nightly build of master. >From the cray box: ommon_ugni.c:30:5: error: 'MCA_BASE_VERSION_2_0_0' undeclared here (not in a function) MCA_BASE_VERSION_2_0_0, common_ugni.c:31:5: warning: in

Re: [OMPI devel] v1.8.5 NEWS and README

Hi Jeff, Kind of sad but I don't want to sign up for XC support for 1.8.5. Cray XK is just an XE but with one opteron socket/node removed and replaced with an nvidia GPU on a daughter card, so I"m willing to sign up for supporting that. So on master README say we support Cray XE, XK, and XC syst

Re: [OMPI devel] Assigning processes to cores 1.4.2, 1.6.4 and 1.8.4

Ok, seems like I am making some progress here. Thanks for the help. I turned HT off. Now I can run v 1.4.2, 1.6.4 and 1.8.4 all compiled the same compiler and run on the same machine 1.4.2 runs this job in 59 minutes. 1.6.4 and 1.8.4 run the job in 1hr 24 minutes. 1.4.2 uses just --mca paffinu

Re: [OMPI devel] Assigning processes to cores 1.4.2, 1.6.4 and 1.8.4

Note where I said "1 hour 14 minutes" it should have read "1 hour 24 minutes"... From: Tom Wurgler Sent: Friday, April 17, 2015 2:14 PM To: Open MPI Developers Subject: Re: [OMPI devel] Assigning processes to cores 1.4.2, 1.6.4 and 1.8.4 Ok, seems like I am maki

Re: [OMPI devel] v1.8.5 NEWS and README

On Fri, Apr 17, 2015 at 5:57 AM, Jeff Squyres (jsquyres) wrote: > - OS X (10.6, 10.7, 10.8, 10.9), 32 and 64 bit (x86_64), with gcc and > Absoft compilers (*) > Since about 10.7 (depending which XCode you installed), cc and c++ have been Clang and Clang++ on Mac OS X. The "gcc" is optional

Re: [OMPI devel] v1.8.5 NEWS and README

Guess I’m puzzled by the XC comment on 1.8.5 given that I’m working with at least one group that is using it on an XC system. If you don’t want to support it, I understand - but we should be clear that it may well work anyway. > On Apr 17, 2015, at 9:28 AM, Howard Pritchard wrote: > > Hi Jeff

Re: [OMPI devel] interaction with slurm 14.11

Hmmm…but what if a user -doesn’t- want their environment forwarded? Seems presumptuous of us to arbitrarily decide to do so on their behalf. > On Apr 16, 2015, at 7:42 PM, David Singleton > wrote: > > > Our site effectively runs all slurm jobs with sbatch --export=NONE ... and > creates th

Re: [OMPI devel] v1.8.5 NEWS and README

On Fri, Apr 17, 2015 at 1:02 PM, Ralph Castain wrote: [...regarding Cray XC...] > If you don't want to support it, I understand - but we should be clear > that it may well work anyway. Ralph, Do you really want to enumerate all of the "it may well work anyway" platforms? If so, I have quite a

Re: [OMPI devel] v1.8.5 NEWS and README

That’s where I was suggesting it be put as opposed to just removing it entirely > On Apr 17, 2015, at 1:16 PM, Paul Hargrove wrote: > > > On Fri, Apr 17, 2015 at 1:02 PM, Ralph Castain > wrote: > [...regarding Cray XC...] > If you don’t want to support it, I understan

Re: [OMPI devel] interaction with slurm 14.11

Ralph, I think David's concern is that because Slurm has changed their default behavior, Open MPI's default behavior has changed as well. The request (on which I have no opinion) appears to be that ORTE make an explicit request for the behavior that was the previous default in Slurm. That would en

Re: [OMPI devel] v1.8.5 NEWS and README

Right on Paul! I can certainly get 1.8.5 to "work" on cray systems like hopper, but last time I tried out of the box I had to fix up the pmi in ess_pmi_module.c because with recent cray PMI's (like the ones now default on hopper), the configure ends up resulting in the use of PMI_KVS_Put/get, whic

Re: [OMPI devel] interaction with slurm 14.11

> On Apr 17, 2015, at 1:27 PM, Paul Hargrove wrote: > > Ralph, > > I think David's concern is that because Slurm has changed their default > behavior, Open MPI's default behavior has changed as well. > The request (on which I have no opinion) appears to be that ORTE make an > explicit request

Re: [OMPI devel] v1.8.5 NEWS and README

On Apr 17, 2015, at 2:19 PM, Paul Hargrove wrote: > > - OS X (10.6, 10.7, 10.8, 10.9), 32 and 64 bit (x86_64), with gcc and > Absoft compilers (*) > > Since about 10.7 (depending which XCode you installed), cc and c++ have been > Clang and Clang++ on Mac OS X. ...snip Good point: - O

Re: [OMPI devel] v1.8.5 NEWS and README

On Apr 17, 2015, at 6:28 PM, Jeff Squyres (jsquyres) wrote: > > - OS X (10.6, 10.7, 10.8, 10.9), 32 and 64 bit (x86_64), with XCode >and Absoft compilers (*) Actually, we should include 10.10 in there. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.c

Re: [OMPI devel] interaction with slurm 14.11

On Sat, Apr 18, 2015 at 6:27 AM, Paul Hargrove wrote: > > The problem here appears to be that the new (--export=NONE) behavior means > that $PATH and/or $LD_LIBRARY_PATH are not propagated, and thus orted could > not be found. > I believe you can configure Open MPI with > --enable-mpirun-prefix-b

Re: [OMPI devel] interaction with slurm 14.11

> On Apr 17, 2015, at 3:54 PM, David Singleton > wrote: > > > > On Sat, Apr 18, 2015 at 6:27 AM, Paul Hargrove > wrote: > > The problem here appears to be that the new (--export=NONE) behavior means > that $PATH and/or $LD_LIBRARY_PATH are not propagated, and thu

Re: [OMPI devel] Assigning processes to cores 1.4.2, 1.6.4 and 1.8.4

Hi Tom Glad you are making some progress! Note that the 1.8 series uses hwloc for its affinity operations, while the 1.4 and 1.6 series used the old plpa code. Hence, you will not find the “affinity” components in the 1.8 ompi_info output. Is there some reason you didn’t compile OMPI on the AMD