Re: [OMPI devel] VampirTrace and MPI_Init_thread()

[Matthias Jurenz]

Hello Lisandro,

this problem will be fixed in the next Open MPI release. There was an obsolete 
preprocessor condition around the MPI_Init_thread wrapper, so the source code 
could never be compiled :-(

Thanks for the hint.

Matthias

On Wednesday 11 August 2010 05:34:33 Lisandro Dalcin wrote:
> On 10 August 2010 22:59, George Bosilca  wrote:
> > Lisandro,
> >
> > Thanks for the report. I quickly checked the Open MPI source code and the
> > .so library and both show the existence of both MPI_Init_thread and
> > PMPI_Init_thread symbols.
> >
> > 00031b60 T _MPI_Init_thread
> > 0005e7c0 T _PMPI_Init_thread
> >
> > I CC'ed the VT folks.
> 
> OK. Now I more confident that the problem is in VT:
> 
> nm /usr/local/openmpi/1.4.2/lib/libvt.mpi.a | grep MPI_Init
> 00ab T MPI_Init
>  U PMPI_Init
> 
> I would expect a "  T MPI_Init_thread" line to appear, but it is
> not the case.
> 
> Many thanks,
> 


smime.p7s
Description: S/MIME cryptographic signature

Re: [OMPI devel] Trunk Commit Heads-up: New Common Shared Memory Component

[Graham, Richard L.]

Is this a modification of the existing component, or a new component ?

Rich


On 8/10/10 10:52 AM, "Samuel K. Gutierrez"  wrote:

Hi,

I wanted to give everyone a heads-up about a new POSIX shared memory
component
that has been in the works for a while now and is ready to be pushed
into the
trunk.

http://bitbucket.org/samuelkgutierrez/ompi_posix_sm_new

Some highlights:
o New posix component now the new default.
o May address some of the shared memory performance issues users
encounter
   when OMPI's session directories are inadvertently placed on a non-
local
   filesystem.
o Silent component failover.
o In the default case, if the posix component fails initialization,
   mmap will be selected.
o The sysv component will only be queried for selection if it is
placed before
the mmap component (for example, -mca mpi_common_sm
sysv,posix,mmap).  In the
default case, sysv will never be queried/selected.
o Per some on-list discussion, now unlinking mmaped file in both mmap
and posix
components (see: "System V Shared Memory for Open MPI: Request for
Community
Input and Testing" thread).
o  Assuming local process homogeneity with respect to all utilized
shared
 memory facilities. That is, if one local process deems a
particular shared
 memory facility acceptable, then ALL local processes should be
able to
 utilize that facility. As it stands, this is an important point
because one
 process dictates to all other local processes which common sm
component will
 be selected based on its own, local run-time test.
o Addressed some of George's code reuse concerns.

If there are no major objections by August 17th, I'll commit the code
after the
Tuesday morning conference call.

Thanks!

--
Samuel K. Gutierrez
Los Alamos National Laboratory





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] Trunk Commit Heads-up: New Common Shared Memory Component

[Samuel K. Gutierrez]

Hi Rich,

It's a modification to the existing common sm component.  The  
modifications do include the addition of a new POSIX shared memory  
facility, however.


Sam

On Aug 11, 2010, at 10:05 AM, Graham, Richard L. wrote:


Is this a modification of the existing component, or a new component ?

Rich


On 8/10/10 10:52 AM, "Samuel K. Gutierrez"  wrote:

Hi,

I wanted to give everyone a heads-up about a new POSIX shared memory
component
that has been in the works for a while now and is ready to be pushed
into the
trunk.

http://bitbucket.org/samuelkgutierrez/ompi_posix_sm_new

Some highlights:
o New posix component now the new default.
   o May address some of the shared memory performance issues  
users

encounter
  when OMPI's session directories are inadvertently placed  
on a non-

local
  filesystem.
o Silent component failover.
   o In the default case, if the posix component fails  
initialization,

  mmap will be selected.
o The sysv component will only be queried for selection if it is
placed before
   the mmap component (for example, -mca mpi_common_sm
sysv,posix,mmap).  In the
   default case, sysv will never be queried/selected.
o Per some on-list discussion, now unlinking mmaped file in both mmap
and posix
   components (see: "System V Shared Memory for Open MPI: Request for
Community
   Input and Testing" thread).
o  Assuming local process homogeneity with respect to all utilized
shared
memory facilities. That is, if one local process deems a
particular shared
memory facility acceptable, then ALL local processes should be
able to
utilize that facility. As it stands, this is an important point
because one
process dictates to all other local processes which common sm
component will
be selected based on its own, local run-time test.
o Addressed some of George's code reuse concerns.

If there are no major objections by August 17th, I'll commit the code
after the
Tuesday morning conference call.

Thanks!

--
Samuel K. Gutierrez
Los Alamos National Laboratory





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

[OMPI devel] Trunk appears broken

[Ralph Castain]

I'm still investigating the rev that caused the break, but I wanted to warn 
everyone that the current trunk is broken:

Ralph:ompi-trunk rhc$ mpirun -n 3 hostname
Ralph
Ralph:ompi-trunk rhc$ 


Note only the one output. I have confirmed that all three processes ran, so 
this appears to be a case of lost IO. Not entirely sure of the reason yet.

Ralph

[OMPI devel] Trunk is okay

[Ralph Castain]

After digging deeper, this appears to have been an issue with corruption on
my computer - the trunk seems to be working on my other computer.

False alarm - I'll let you know if it recurs under further investigation.

Re: [OMPI devel] Trunk Commit Heads-up: New Common Shared Memory Component

[Terry Dontje]

I've done some minor testing on Linux looking at resident and shared 
memory sizes for np=4, 8 and 16 jobs.  I could not see any appreciable 
differences in sizes in the process between sysv, posix or mmap usage in 
the SM btl.


So I am still somewhat non-plussed about making this the default.  It 
seems like the biggest gain out of using posix might be one doesn't need 
to worry about the location of the backing file.  This seems kind of 
frivolous to me since there is a warning that happens if the backing 
file is not memory based.


I still working on testing the code on Solaris but I don't imagine I 
will see anything that will change my mind.


--td

Samuel K. Gutierrez wrote:

Hi Rich,

It's a modification to the existing common sm component.  The 
modifications do include the addition of a new POSIX shared memory 
facility, however.


Sam

On Aug 11, 2010, at 10:05 AM, Graham, Richard L. wrote:


Is this a modification of the existing component, or a new component ?

Rich


On 8/10/10 10:52 AM, "Samuel K. Gutierrez"  wrote:

Hi,

I wanted to give everyone a heads-up about a new POSIX shared memory
component
that has been in the works for a while now and is ready to be pushed
into the
trunk.

http://bitbucket.org/samuelkgutierrez/ompi_posix_sm_new

Some highlights:
o New posix component now the new default.
   o May address some of the shared memory performance issues users
encounter
  when OMPI's session directories are inadvertently placed on 
a non-

local
  filesystem.
o Silent component failover.
   o In the default case, if the posix component fails 
initialization,

  mmap will be selected.
o The sysv component will only be queried for selection if it is
placed before
   the mmap component (for example, -mca mpi_common_sm
sysv,posix,mmap).  In the
   default case, sysv will never be queried/selected.
o Per some on-list discussion, now unlinking mmaped file in both mmap
and posix
   components (see: "System V Shared Memory for Open MPI: Request for
Community
   Input and Testing" thread).
o  Assuming local process homogeneity with respect to all utilized
shared
memory facilities. That is, if one local process deems a
particular shared
memory facility acceptable, then ALL local processes should be
able to
utilize that facility. As it stands, this is an important point
because one
process dictates to all other local processes which common sm
component will
be selected based on its own, local run-time test.
o Addressed some of George's code reuse concerns.

If there are no major objections by August 17th, I'll commit the code
after the
Tuesday morning conference call.

Thanks!

--
Samuel K. Gutierrez
Los Alamos National Laboratory





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Oracle
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.650.633.7054
Oracle * - Performance Technologies*
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com 

Re: [OMPI devel] Trunk Commit Heads-up: New Common Shared Memory Component

[Samuel K. Gutierrez]

Hi Terry,








On Aug 11, 2010, at 12:34 PM, Terry Dontje wrote:

I've done some minor testing on Linux looking at resident and shared  
memory sizes for np=4, 8 and 16 jobs.  I could not see any  
appreciable differences in sizes in the process between sysv, posix  
or mmap usage in the SM btl.


So I am still somewhat non-plussed about making this the default.   
It seems like the biggest gain out of using posix might be one  
doesn't need to worry about the location of the backing file.  This  
seems kind of frivolous to me since there is a warning that happens  
if the backing file is not memory based.


If I'm not mistaken, the warning is only issued if the backing files  
is stored on the following file systems: Lustre, NFS, Panasas, and  
GPFS  (see: opal_path_nfs in opal/util/path.c).  Based on the  
performance numbers that Sylvain provided on June 9th of this year  
(see attached),  there was a performance difference between mmap on / 
tmp and mmap on a tmpfs-like file system (/dev/shm in that particular  
case).  Using the new POSIX component provides us with a portable way  
to provide similar shared memory performance gains without having to  
worry about where the OMPI session directory is rooted.


--
Samuel K. Gutierrez
Los Alamos National Laboratory





I still working on testing the code on Solaris but I don't imagine I  
will see anything that will change my mind.


--td

Samuel K. Gutierrez wrote:


Hi Rich,

It's a modification to the existing common sm component.  The  
modifications do include the addition of a new POSIX shared memory  
facility, however.


Sam

On Aug 11, 2010, at 10:05 AM, Graham, Richard L. wrote:

Is this a modification of the existing component, or a new  
component ?


Rich


On 8/10/10 10:52 AM, "Samuel K. Gutierrez"  wrote:

Hi,

I wanted to give everyone a heads-up about a new POSIX shared memory
component
that has been in the works for a while now and is ready to be pushed
into the
trunk.

http://bitbucket.org/samuelkgutierrez/ompi_posix_sm_new

Some highlights:
o New posix component now the new default.
   o May address some of the shared memory performance issues  
users

encounter
  when OMPI's session directories are inadvertently placed  
on a non-

local
  filesystem.
o Silent component failover.
   o In the default case, if the posix component fails  
initialization,

  mmap will be selected.
o The sysv component will only be queried for selection if it is
placed before
   the mmap component (for example, -mca mpi_common_sm
sysv,posix,mmap).  In the
   default case, sysv will never be queried/selected.
o Per some on-list discussion, now unlinking mmaped file in both  
mmap

and posix
   components (see: "System V Shared Memory for Open MPI: Request  
for

Community
   Input and Testing" thread).
o  Assuming local process homogeneity with respect to all utilized
shared
memory facilities. That is, if one local process deems a
particular shared
memory facility acceptable, then ALL local processes should be
able to
utilize that facility. As it stands, this is an important point
because one
process dictates to all other local processes which common sm
component will
be selected based on its own, local run-time test.
o Addressed some of George's code reuse concerns.

If there are no major objections by August 17th, I'll commit the  
code

after the
Tuesday morning conference call.

Thanks!

--
Samuel K. Gutierrez
Los Alamos National Laboratory





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--

Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.650.633.7054
Oracle - Performance Technologies
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] Trunk Commit Heads-up: New Common Shared Memory Component

[Samuel K. Gutierrez]

Hi Terry,

One more thing...  Before testing on Solaris 10, could you please  
update (I just committed a Solaris 10 fix).


Thanks,

--
Samuel K. Gutierrez
Los Alamos National Laboratory

On Aug 11, 2010, at 1:15 PM, Samuel K. Gutierrez wrote:


Hi Terry,








On Aug 11, 2010, at 12:34 PM, Terry Dontje wrote:

I've done some minor testing on Linux looking at resident and  
shared memory sizes for np=4, 8 and 16 jobs.  I could not see any  
appreciable differences in sizes in the process between sysv, posix  
or mmap usage in the SM btl.


So I am still somewhat non-plussed about making this the default.   
It seems like the biggest gain out of using posix might be one  
doesn't need to worry about the location of the backing file.  This  
seems kind of frivolous to me since there is a warning that happens  
if the backing file is not memory based.


If I'm not mistaken, the warning is only issued if the backing files  
is stored on the following file systems: Lustre, NFS, Panasas, and  
GPFS  (see: opal_path_nfs in opal/util/path.c).  Based on the  
performance numbers that Sylvain provided on June 9th of this year  
(see attached),  there was a performance difference between mmap on / 
tmp and mmap on a tmpfs-like file system (/dev/shm in that  
particular case).  Using the new POSIX component provides us with a  
portable way to provide similar shared memory performance gains  
without having to worry about where the OMPI session directory is  
rooted.


--
Samuel K. Gutierrez
Los Alamos National Laboratory





I still working on testing the code on Solaris but I don't imagine  
I will see anything that will change my mind.


--td

Samuel K. Gutierrez wrote:


Hi Rich,

It's a modification to the existing common sm component.  The  
modifications do include the addition of a new POSIX shared memory  
facility, however.


Sam

On Aug 11, 2010, at 10:05 AM, Graham, Richard L. wrote:

Is this a modification of the existing component, or a new  
component ?


Rich


On 8/10/10 10:52 AM, "Samuel K. Gutierrez"  wrote:

Hi,

I wanted to give everyone a heads-up about a new POSIX shared  
memory

component
that has been in the works for a while now and is ready to be  
pushed

into the
trunk.

http://bitbucket.org/samuelkgutierrez/ompi_posix_sm_new

Some highlights:
o New posix component now the new default.
   o May address some of the shared memory performance issues  
users

encounter
  when OMPI's session directories are inadvertently  
placed on a non-

local
  filesystem.
o Silent component failover.
   o In the default case, if the posix component fails  
initialization,

  mmap will be selected.
o The sysv component will only be queried for selection if it is
placed before
   the mmap component (for example, -mca mpi_common_sm
sysv,posix,mmap).  In the
   default case, sysv will never be queried/selected.
o Per some on-list discussion, now unlinking mmaped file in both  
mmap

and posix
   components (see: "System V Shared Memory for Open MPI: Request  
for

Community
   Input and Testing" thread).
o  Assuming local process homogeneity with respect to all utilized
shared
memory facilities. That is, if one local process deems a
particular shared
memory facility acceptable, then ALL local processes should be
able to
utilize that facility. As it stands, this is an important point
because one
process dictates to all other local processes which common sm
component will
be selected based on its own, local run-time test.
o Addressed some of George's code reuse concerns.

If there are no major objections by August 17th, I'll commit the  
code

after the
Tuesday morning conference call.

Thanks!

--
Samuel K. Gutierrez
Los Alamos National Laboratory





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--

Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.650.633.7054
Oracle - Performance Technologies
95 Network Drive, Burlington, MA 01803
Email terry.don...@oracle.com

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] VampirTrace and MPI_Init_thread()

[Lisandro Dalcin]

On 11 August 2010 03:12, Matthias Jurenz  wrote:
> Hello Lisandro,
>
> this problem will be fixed in the next Open MPI release. There was an obsolete
> preprocessor condition around the MPI_Init_thread wrapper, so the source code
> could never be compiled :-(
>
> Thanks for the hint.
>
> Matthias
>

OK. Many thanks for you clarification.

BTW, I have and additional issue. I'm trying to build as shared
library from libvt*.a using by passing -whole-archive to the linker.
The idea behind this is to use that library with LD_PRELOAD to get MPI
tracing of a binary compiled with plain mpicc (i.e, not mpicc-vt). For
example, I managed to get this trick working with MPE. Moreover, I can
enable MPI profiling at runtime in a Python script using mpi4pt by
dlopen'ing the shared lib with profiling symbols before loading the
mpi4py.MPI Python extension module. Being able to profile without a
recompile is nice ;-)

However, see this:

$ pwd
/usr/local/openmpi/1.4.2/lib

$ ll libvt*
-rw-r--r--. 1 root root 410784 2010-05-05 20:40 libvt.a
-rw-r--r--. 1 root root 197618 2010-05-05 20:40 libvt.fmpi.a
-rw-r--r--. 1 root root 569128 2010-05-05 20:40 libvt.mpi.a
-rw-r--r--. 1 root root 503514 2010-05-05 20:40 libvt.omp.a
-rw-r--r--. 1 root root 661466 2010-05-05 20:40 libvt.ompi.a

$ nm libvt* | grep pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table
 U pomp_rd_table

That symbol (and possibly others) are undefined and I cannot found
them elsewhere. Is there any easy way to build a shared lib with the
MPI_xxx symbols?


-- 
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169