[OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Tobias Hilbrich
Hi all, a wonderful puzzle for the OSX folks in your team (Reproducer attached): Attached source file builds with Xcode 7.0.0, but fails since the release of Xcode 7.0.1. I post it here since it results from an "if (MPI_COMM_NULL == MPI_COMM_SELF)". I am actually curious whether this is somehow

Re: [OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Gilles Gouaillardet
Tobias, Fwiw, MPI_Comm_compare can be used to compare communicators. Hopefully, this is also compiler friendly. Cheers, Gilles On Tuesday, October 20, 2015, Tobias Hilbrich wrote: > Hi all, > > a wonderful puzzle for the OSX folks in your team (Reproducer attached): > > Attached source file b

Re: [OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Gilles Gouaillardet
Tobias, Btw, did you recompile ompi with this xcode ? Iirc, we do similar comparisons in ompi itself Cheers, Gilles Tobias Hilbrich wrote: >Hi all, > >a wonderful puzzle for the OSX folks in your team (Reproducer attached): > >Attached source file builds with Xcode 7.0.0, but fails since the r

Re: [OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Tobias Hilbrich
Hi Gilles, yes MPI_Comm_compare can solve the issue, though one has to take care not to pass MPI_COMM_NULL into that (according the MPICH documentation at least). Even a simple modification can remove the issue: " ... MPI_Comm comm = MPI_COMM_NULL; if (MPI_COMM_NULL == comm) ... " Attached an

Re: [OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Tobias Hilbrich
Hi Gilles, as for that, recompiling OpenMPI works, but causes no change here. -Tobias -- Dr.-Ing. Tobias Hilbrich Research Assistant Technische Universitaet Dresden, Germany Tel.: +49 (351) 463-38485 E-Mail: tobias.hilbr...@tu-dresden.de > On 20 Oct 2015, at 11:42, Gilles Gouaillardet > wro

Re: [OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Jeff Squyres (jsquyres)
Tobias -- I'm able to replicate the problem with 1.8.4 and 1.8.8 and 1.10.0 on OS X 10.10.5 (Yosemite) and Xcode 7.0.1. I'd suggest filing a bug at bugreporter.apple.com with the information that clang provided in their output. If there's a bug in the compiler, there's not much that we here a

[OMPI devel] IBM Innovation Center Reserved for Open MPI Face-2-Face

2015-10-20 Thread Geoffrey Paulsen
We have the Dallas IBM Innovation Center (http://ibm.com/partnerworld/iic/dallas.htm) reserved 2/23 - 2/25, 2016.   IBM Innovation Center - Dallas1177 South Beltline RdCoppell, TX 75019469-549-8444   https://www.google.com/maps/place/IBM+Innovation+Center+-+Dallas/@32.942725,-96.9965226,17z/data=""

[OMPI devel] Open MPI State of the Union BOF @SC15

2015-10-20 Thread Jeff Squyres (jsquyres)
We're about T-1 month away from the Open MPI State of the Union BOF at SC'15. It's at 12:15pm on Wednesday, 18 Nov, 2015, in room 18C/D: http://sc15.supercomputing.org/schedule/event_detail?evid=bof107 As usual, since we only have an hour in the BOF, we like to capture some of your questions

Re: [OMPI devel] IBM Innovation Center Reserved for Open MPI Face-2-Face

2015-10-20 Thread Jeff Squyres (jsquyres)
Schweet -- thanks IBM! We have therefore settled on the final dates for the meeting: Feb Tue-Thu, Feb 23-25, 2015. Wiki page has been updated: https://github.com/open-mpi/ompi/wiki/Meeting-2016-02 > On Oct 20, 2015, at 10:35 AM, Geoffrey Paulsen wrote: > > We have the Dallas IBM Innova

[OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
We talked about this on the call last week. I'm guessing we'll talk about this at the Feb dev meeting, but we need to think about this a bit before hand. Here's a little more fuel for the fire: let's at least specify the problem space a bit more precisely... (this item is on the agenda for the

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Paul Hargrove
I looked quickly over the quoted emails and didn't see something I had hoped/expected to. In addition to the "dimensions" of type, api and pml I think users may also be concerned about the "port" dimension (or device if you prefer). So, it might be worth including that in the discussion of the hig

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Ralph Castain
Don’t you also have the question of, for example, PSM via the mtl/psm versus PSM via the mtl/ofi path? So I think you need to split the entries in #2 as: PSM/MTL PSM/MTL/OFI PSM2/MTL PSM2/MTL/OFI etc. Or we could remove the PSM/PSM2 MTL components and just drive those thru the OFI provider int

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
On Oct 20, 2015, at 3:42 PM, Jeff Squyres (jsquyres) wrote: > > I'm guessing we'll talk about this at the Feb dev meeting, but we need to > think about this a bit before hand. Here's a little more fuel for the fire: > let's at least specify the problem space a bit more precisely... I'm replyi

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
On Oct 20, 2015, at 4:35 PM, Paul Hargrove wrote: > > As an example, I might have two ethernet cards, one of which is a Cisco VNIC. > I would want be able to control which BTL or MTL is used on those NICs > independently, including the option to disable use of one or the other. > I do not want t

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Ralph Castain
Your last point about the qualifiers is kinda what I was hinting at in my note. If you have usnic support via the OFi MTL, why do you also need it as a BTL? The BTL needs libfabric anyway, yes? So is there some value in having both methods? Same question for PSM and PSM2, and probably others I

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
On Oct 20, 2015, at 4:49 PM, Ralph Castain wrote: > > Your last point about the qualifiers is kinda what I was hinting at in my > note. If you have usnic support via the OFi MTL, why do you also need it as a > BTL? The BTL needs libfabric anyway, yes? So is there some value in having > both me

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Ralph Castain
Understood - but can we narrow it down a bit? Specifically, do we need both BTL and MTL access to the same network? This would cut the combinations by 2x right away. Then we could potentially remove the network-specific MTLs. Then we just have to deal with UCX vs libfabric - so only the two deci

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Paul Hargrove
On Tue, Oct 20, 2015 at 1:47 PM, Jeff Squyres (jsquyres) wrote: > On Oct 20, 2015, at 4:35 PM, Paul Hargrove wrote: > > > > As an example, I might have two ethernet cards, one of which is a Cisco > VNIC. > > I would want be able to control which BTL or MTL is used on those NICs > independently,

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
On Oct 20, 2015, at 5:26 PM, Ralph Castain wrote: > > Understood - but can we narrow it down a bit? Specifically, do we need both > BTL and MTL access to the same network? That's what I'm saying: for usnic, I don't know yet (we haven't finished our tag matching implementation yet). I suspect

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
On Oct 20, 2015, at 5:26 PM, Paul Hargrove wrote: > > I think heterogeneous multirail is still pretty uncommon. It might still be > ok to force users (or better yet, their admins -- via the global > mca-params.conf file) to use level 3 to precisely specify which network / > OMPI API to use (e

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Jeff Squyres (jsquyres)
On Oct 20, 2015, at 5:26 PM, Paul Hargrove wrote: > > I have multiple ports of the same type, lets say a dual-port Mellanox HCA, > and just want to disable one of them (reserving it for Luster perhaps). > If OMPI is hiding from me the details of the API selection, how do I > enable/disable spec

Re: [OMPI devel] Specifying networks/APIs for OMPI (was: topic for agenda)

2015-10-20 Thread Paul Hargrove
On Tue, Oct 20, 2015 at 3:00 PM, Jeff Squyres (jsquyres) wrote: > I don't think it gets up to 7 MCA params to guarantee a specific API path > is used to get to a specific network / port, but your overall point is fair. Jeff, it sounds to me you are responding to a different problem than the one

Re: [OMPI devel] orte-dvm / orte-submit race condition

2015-10-20 Thread Ralph Castain
Hey Mark Can you do me a favor? I’m totally buried, but I have been able to replicate this on my machine, so it is a definite race condition. What would really help me is if you could do the following: * start the orte-dvm with the “—mca state_base_verbose 10” option, and capture stdout/stderr

Re: [OMPI devel] Issue with OpenMPI 1.8.4 + Xcode 7.0.1

2015-10-20 Thread Gilles Gouaillardet
Tobias and Jeff, here is attached a simple reproducer. i reported it to apple with id 23195613 /* you need to replace #define EXTERN with #define EXTERN extern so the compilation successes with an other compiler (otherwise, compilation fails but compiler does not crash) */ Cheers, Gilles On T

[OMPI devel] Problem running openmpi on nodes connected via eth

2015-10-20 Thread Andrej Prsa
Hi everyone, We have a small cluster of 6 identical 48-core nodes for astrophysical research. We are struggling on getting openmpi to run efficiently on the nodes. The head node is running ubuntu and openmpi-1.6.5 on a local disk. All worker nodes are booting from NFS exported root that resides on

Re: [OMPI devel] Problem running openmpi on nodes connected via eth

2015-10-20 Thread Gilles Gouaillardet
Andrej, by "running on the head node", shall i understand you mean "running mpirun command *and* all mpi tasks on the head node" ? by "running on the compute node", shall i understand you mean "running mpirun on the compute node *and* all mpi tasks on the *same* compute node" ? or do you mean

Re: [OMPI devel] Problem running openmpi on nodes connected via eth

2015-10-20 Thread Andrej Prsa
Hi Gilles, Thanks for your reply! > by "running on the head node", shall i understand you mean > "running mpirun command *and* all mpi tasks on the head node" ? Precisely. > by "running on the compute node", shall i understand you mean > "running mpirun on the compute node *and* all mpi tasks o