Re: [OMPI devel] How to deal with F90 mpi.mod with single stack and multiple compiler suites?

[Larry Baker]

Chris,

.mod files are compiler-specific, and may even be version-specific.  You may, 
however, be lucky enough to compile the Fortran interface definitions with 
ifort and supply that mpi.mod to ifort, even though the actual code was 
compiled with gfortran.  I have never tried that -- we build separate 
development trees for each OpenMPI version/compiler product/compiler version.

Larry Baker
US Geological Survey
650-329-5608
ba...@usgs.gov



On 22 Aug 2013, at 6:50 AM, Jeff Squyres (jsquyres) wrote:

> Sadly, probably not. :(. You'll prbably have the same problem with c++, too. 
> 
> There *may* be compatibility command line options for ifort/icpc to make them 
> link compatible w gfortran/g++, but I've never had much faith in them. 
> 
> Sent from my phone. No type good. 
> 
> On Aug 22, 2013, at 2:24 AM, "Christopher Samuel"  
> wrote:
> 
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>> 
>> Hi folks,
>> 
>> We've got what we thought would be a fairly standard OMPI (1.6.5)
>> install which is a single install built with GCC and then setting the
>> appropriate variables to use the Intel compilers when someone loads
>> our "intel" module:
>> 
>> $ module show intel
>> [...]
>> setenv   OMPI_CC icc
>> setenv   OMPI_CXX icpc
>> setenv   OMPI_F77 ifort
>> setenv   OMPI_FC ifort
>> setenv   OMPI_CFLAGS -xHOST -O3 -mkl=sequential
>> setenv   OMPI_FFLAGS -xHOST -O3 -mkl=sequential
>> setenv   OMPI_FCFLAGS -xHOST -O3 -mkl=sequential
>> setenv   OMPI_CXXFLAGS -xHOST -O3 -mkl=sequential
>> 
>> This works wonderfully, *except* when our director attempted to build
>> an F90 program with the Intel compilers that fails to build because
>> the mpi.mod F90 module was produced with gfortran rather than the
>> Intel compilers. :-(
>> 
>> Is there any way to avoid having to do parallel installs of OMPI with
>> GCC and Intel compilers just to have two different versions of these
>> files?
>> 
>> My brief googling hasn't indicated anything, and I don't see anything
>> in the mpif90 manual page (though I have to admit I've had to rush to
>> try and get this done before I need to leave for the day). :-(
>> 
>> cheers,
>> Chris
>> - -- 
>> Christopher SamuelSenior Systems Administrator
>> VLSCI - Victorian Life Sciences Computation Initiative
>> Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
>> http://www.vlsci.org.au/  http://twitter.com/vlsci
>> 
>> -BEGIN PGP SIGNATURE-
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>> 
>> iEYEARECAAYFAlIVrpIACgkQO2KABBYQAh/GAQCggQGnc18kSfMcGle3a3pWZGgD
>> UQ8AoIz61uuOPj+TFJwSYMTaAtUBLk3J
>> =yJ6J
>> -END PGP SIGNATURE-
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r29055 - in trunk/ompi/mca: btl btl/smcuda common/cuda pml/ob1

[Rolf vandeVaart]

Hi George:

The reason it tainted the PML is because the CUDA IPC support makes use of the 
large message RDMA protocol of the PML layer.  The smcuda btl starts up, but 
does not initially support any large message RDMA (RGET,RPUT) protocols.  Then 
when a GPU buffer is first accessed, the smcuda btl starts an exchange of some 
control messages with its peer.  If they determine that they can support CUDA 
IPC, then the smcuda calls up into the PML layer and says it is OK to start 
using the large message RDMA.  This all happens in code that is only compiled 
in if the user asks for CUDA-aware support.

The key requirement was I wanted to dynamically add the support for CUDA IPC 
when the user first started accessing GPU buffers rather than during MPI_Init.
This the best way I could figure out how to accomplish this but I am open to 
other ideas.   

Thanks,
Rolf

>-Original Message-
>From: devel [mailto:devel-boun...@open-mpi.org] On Behalf Of George
>Bosilca
>Sent: Thursday, August 22, 2013 11:32 AM
>To: de...@open-mpi.org
>Subject: Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r29055 - in
>trunk/ompi/mca: btl btl/smcuda common/cuda pml/ob1
>
>I'm not very keen of seeing BTL modification tainting the PML. I would have
>expected support for IPC between GPU must be a BTL-level decision, no a
>special path in the PML.
>
>Is there a reason IPC support cannot be hidden down in the SMCUDA BTL?
>
>  Thanks,
>George.
>
>On Aug 21, 2013, at 23:00 , svn-commit-mai...@open-mpi.org wrote:
>
>> Author: rolfv (Rolf Vandevaart)
>> Date: 2013-08-21 17:00:09 EDT (Wed, 21 Aug 2013) New Revision: 29055
>> URL: https://svn.open-mpi.org/trac/ompi/changeset/29055
>>
>> Log:
>> Fix support in smcuda btl so it does not blow up when there is no CUDA IPC
>support between two GPUs. Also make it so CUDA IPC support is added
>dynamically.
>> Fixes ticket 3531.
>>
>> Added:
>>   trunk/ompi/mca/btl/smcuda/README
>> Text files modified:
>>   trunk/ompi/mca/btl/btl.h | 2
>>   trunk/ompi/mca/btl/smcuda/README |   113
>++
>>   trunk/ompi/mca/btl/smcuda/btl_smcuda.c   |   104
>
>>   trunk/ompi/mca/btl/smcuda/btl_smcuda.h   |28 +
>>   trunk/ompi/mca/btl/smcuda/btl_smcuda_component.c |   200
>+++
>>   trunk/ompi/mca/btl/smcuda/btl_smcuda_endpoint.h  | 5 +
>>   trunk/ompi/mca/common/cuda/common_cuda.c |29 +
>>   trunk/ompi/mca/common/cuda/common_cuda.h | 3
>>   trunk/ompi/mca/pml/ob1/pml_ob1.c |11 ++
>>   trunk/ompi/mca/pml/ob1/pml_ob1_cuda.c|42 
>>   trunk/ompi/mca/pml/ob1/pml_ob1_recvreq.c | 6
>>   11 files changed, 535 insertions(+), 8 deletions(-)
>
>___
>devel mailing list
>de...@open-mpi.org
>http://www.open-mpi.org/mailman/listinfo.cgi/devel
---
This email message is for the sole use of the intended recipient(s) and may 
contain
confidential information.  Any unauthorized review, use, disclosure or 
distribution
is prohibited.  If you are not the intended recipient, please contact the 
sender by
reply email and destroy all copies of the original message.
---

Re: [OMPI devel] bitbucket mirror out of sync ?

[Guillaume Papauré]

It is ! Thanks !
Le 22/08/2013 15:49, Jeff Squyres (jsquyres) a écrit :

Looks like it's updated now.

On Aug 22, 2013, at 8:44 AM, Jeff Squyres  wrote:


Oops -- I'll bet this got borked when our hosting provider moved servers 
recently.  I'll check; thanks for the heads up!


On Aug 22, 2013, at 4:22 AM, Guillaume Papauré  
wrote:


Hello,

It seems that the openmpi mercurial mirror 
(https://bitbucket.org/ompiteam/ompi-svn-mirror) is not up to date with the svn 
reposiory. According to the history the last hg commit is dated from August 6th 
(svn r29008), contrary to the svn r29056 from yesterday.
Did something bad happened to the update process ?

Regards,
Guillaume


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[OMPI devel] Potential Performance issues with mmap'ed files

[Tim Mattox]

Hello Open MPI developers,
I've been away from the Open MPI code for a long time now, but I just
ran across this article that should give developers second thoughts
about using mmap'ed files in performance critical situations.

"Deferring mtime and ctime updates"
http://lwn.net/SubscriberLink/564120/bdc189d08c512761/

I might suggest an audit of the Open MPI code base for mmap calls
to make sure they are all needed, or if they can be done
a different way, especially in the coll and sm btl.
Yes, the article is about a patchset to lessen the performance problem,
but until that gets into the kernel, and into the distributions, and into
the parallel machines, this could be a source of lost performance in Open MPI.

I seem to recall some work was done after I left development to
make use of POSIX or SYSV shared memory system calls that avoided
having an associated backing-store file.  But I don't remember if
those became the default, or if mmap'ed files was still the default.

Anyway, back to lurking on the mailing lists for me.
P.S. - Darn, my timat...@open-mpi.org e-mail alias is gone... c'est la vie!
-- 
Tim Mattox, Ph.D. - tmat...@gmail.com

Re: [OMPI devel] How to deal with F90 mpi.mod with single stack and multiple compiler suites?

[Jeff Squyres (jsquyres)]

Sadly, probably not. :(. You'll prbably have the same problem with c++, too. 

There *may* be compatibility command line options for ifort/icpc to make them 
link compatible w gfortran/g++, but I've never had much faith in them. 

Sent from my phone. No type good. 

On Aug 22, 2013, at 2:24 AM, "Christopher Samuel"  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hi folks,
> 
> We've got what we thought would be a fairly standard OMPI (1.6.5)
> install which is a single install built with GCC and then setting the
> appropriate variables to use the Intel compilers when someone loads
> our "intel" module:
> 
> $ module show intel
> [...]
> setenv   OMPI_CC icc
> setenv   OMPI_CXX icpc
> setenv   OMPI_F77 ifort
> setenv   OMPI_FC ifort
> setenv   OMPI_CFLAGS -xHOST -O3 -mkl=sequential
> setenv   OMPI_FFLAGS -xHOST -O3 -mkl=sequential
> setenv   OMPI_FCFLAGS -xHOST -O3 -mkl=sequential
> setenv   OMPI_CXXFLAGS -xHOST -O3 -mkl=sequential
> 
> This works wonderfully, *except* when our director attempted to build
> an F90 program with the Intel compilers that fails to build because
> the mpi.mod F90 module was produced with gfortran rather than the
> Intel compilers. :-(
> 
> Is there any way to avoid having to do parallel installs of OMPI with
> GCC and Intel compilers just to have two different versions of these
> files?
> 
> My brief googling hasn't indicated anything, and I don't see anything
> in the mpif90 manual page (though I have to admit I've had to rush to
> try and get this done before I need to leave for the day). :-(
> 
> cheers,
> Chris
> - -- 
> Christopher SamuelSenior Systems Administrator
> VLSCI - Victorian Life Sciences Computation Initiative
> Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
> http://www.vlsci.org.au/  http://twitter.com/vlsci
> 
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> 
> iEYEARECAAYFAlIVrpIACgkQO2KABBYQAh/GAQCggQGnc18kSfMcGle3a3pWZGgD
> UQ8AoIz61uuOPj+TFJwSYMTaAtUBLk3J
> =yJ6J
> -END PGP SIGNATURE-
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] bitbucket mirror out of sync ?

[Jeff Squyres (jsquyres)]

Looks like it's updated now.

On Aug 22, 2013, at 8:44 AM, Jeff Squyres  wrote:

> Oops -- I'll bet this got borked when our hosting provider moved servers 
> recently.  I'll check; thanks for the heads up!
> 
> 
> On Aug 22, 2013, at 4:22 AM, Guillaume Papauré 
>  wrote:
> 
>> Hello,
>> 
>> It seems that the openmpi mercurial mirror 
>> (https://bitbucket.org/ompiteam/ompi-svn-mirror) is not up to date with the 
>> svn reposiory. According to the history the last hg commit is dated from 
>> August 6th (svn r29008), contrary to the svn r29056 from yesterday.
>> Did something bad happened to the update process ?
>> 
>> Regards,
>> Guillaume
>> 
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] bitbucket mirror out of sync ?

[Jeff Squyres (jsquyres)]

Oops -- I'll bet this got borked when our hosting provider moved servers 
recently.  I'll check; thanks for the heads up!


On Aug 22, 2013, at 4:22 AM, Guillaume Papauré  
wrote:

> Hello,
> 
> It seems that the openmpi mercurial mirror 
> (https://bitbucket.org/ompiteam/ompi-svn-mirror) is not up to date with the 
> svn reposiory. According to the history the last hg commit is dated from 
> August 6th (svn r29008), contrary to the svn r29056 from yesterday.
> Did something bad happened to the update process ?
> 
> Regards,
> Guillaume
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] Detailed documentation on OpenMPI

[Jeff Squyres (jsquyres)]

Note that according to http://www.ohloh.net/p/openmpi, OMPI is over 3/4M lines 
of code.  I doubt you will be able to get a thorough understanding of *all* of 
OMPI in a semester or two -- indeed, I doubt that any one Open MPI core 
developer has a thorough understanding of the whole code base (I know I don't!).

I don't say these things to discourage you; I actually just want to encourage 
you to set realistic expectations for what you'll be able to do.

Are there any particular areas that you'd like to focus on?  MPI -- and the 
Open MPI implementation -- covers a *lot* of different subsystems and concepts.


On Aug 22, 2013, at 8:06 AM, mahesh  wrote:

> Hi,
> I am an   newbie to all MPI concepts and I would like to understand the 
> MPI source code thoroughly for
> an academic project. So, what I   need is an detailed explanation of how 
> every framework and module 
> works. It would   be really helpful if wise people could point me to 
> right direction.
> 
> Thanks,
> Mahesh
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[OMPI devel] Detailed documentation on OpenMPI

[mahesh]

Hi,
I am an newbie to all MPI concepts and I would like to understand the MPI 
source code thoroughly for
an academic project. So, what I need is an detailed explanation of how every 
framework and module 
works. It would be really helpful if wise people could point me to right 
direction.

Thanks,
Mahesh

[OMPI devel] bitbucket mirror out of sync ?

[Guillaume Papauré]

Hello,

It seems that the openmpi mercurial mirror 
(https://bitbucket.org/ompiteam/ompi-svn-mirror) is not up to date with 
the svn reposiory. According to the history the last hg commit is dated 
from August 6th (svn r29008), contrary to the svn r29056 from yesterday.

Did something bad happened to the update process ?

Regards,
Guillaume

[OMPI devel] How to deal with F90 mpi.mod with single stack and multiple compiler suites?

[Christopher Samuel]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi folks,

We've got what we thought would be a fairly standard OMPI (1.6.5)
install which is a single install built with GCC and then setting the
appropriate variables to use the Intel compilers when someone loads
our "intel" module:

$ module show intel
[...]
setenv   OMPI_CC icc
setenv   OMPI_CXX icpc
setenv   OMPI_F77 ifort
setenv   OMPI_FC ifort
setenv   OMPI_CFLAGS -xHOST -O3 -mkl=sequential
setenv   OMPI_FFLAGS -xHOST -O3 -mkl=sequential
setenv   OMPI_FCFLAGS -xHOST -O3 -mkl=sequential
setenv   OMPI_CXXFLAGS -xHOST -O3 -mkl=sequential

This works wonderfully, *except* when our director attempted to build
an F90 program with the Intel compilers that fails to build because
the mpi.mod F90 module was produced with gfortran rather than the
Intel compilers. :-(

Is there any way to avoid having to do parallel installs of OMPI with
GCC and Intel compilers just to have two different versions of these
files?

My brief googling hasn't indicated anything, and I don't see anything
in the mpif90 manual page (though I have to admit I've had to rush to
try and get this done before I need to leave for the day). :-(

cheers,
Chris
- -- 
 Christopher SamuelSenior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/  http://twitter.com/vlsci

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlIVrpIACgkQO2KABBYQAh/GAQCggQGnc18kSfMcGle3a3pWZGgD
UQ8AoIz61uuOPj+TFJwSYMTaAtUBLk3J
=yJ6J
-END PGP SIGNATURE-