Re: [OMPI devel] Tarball distribution

[Ralf Wildenhues]

Hi Jeff,

yes, you should definitely do that.

Thanks,
Ralf

* Jeff Squyres wrote on Mon, Jun 27, 2011 at 03:58:14PM CEST:
> We lock in a specific set of Autotools for a given release series and try 
> very hard not to change them for the life of that series, just to prevent 
> unexpected incompatibilities / issues.
> 
> The v1.4 series is using:
> 
> AC 2.63
> AM 1.10.1
> LT 2.2.6b
> m4 1.4.11
> 
> The v1.5 series is using:
> 
> AC 2.65
> AM 1.11
> LT 2.2.6b
> m4 1.4.13
> 
> The upgrade to AM 1.10.3 (for the v1.4 series) and 1.11.1 (for the v1.5 
> series) may be tolerable.  I'll check it out.
> 
> 
> 
> On Jun 27, 2011, at 2:09 AM, Ralf Wildenhues wrote:
> 
> > Hi John,
> > 
> > * John Esmet wrote on Sun, Jun 26, 2011 at 06:41:48AM CEST:
> >> I went to untar the source code and the folders are mode 777. Call me OCD,
> >> but I find this a little strange. What's up?
> > 
> > Newer Automake releases should have that fixed:
> > http://thread.gmane.org/gmane.comp.sysutils.autotools.announce/131
> > 
> > Open MPI should use such a newer Automake for their release process.
> > 
> > Cheers,
> > Ralf

Re: [OMPI devel] Tarball distribution

[Ralf Wildenhues]

Hi John,

* John Esmet wrote on Sun, Jun 26, 2011 at 06:41:48AM CEST:
> I went to untar the source code and the folders are mode 777. Call me OCD,
> but I find this a little strange. What's up?

Newer Automake releases should have that fixed:
http://thread.gmane.org/gmane.comp.sysutils.autotools.announce/131

Open MPI should use such a newer Automake for their release process.

Cheers,
Ralf

Re: [OMPI devel] Compiling problem in trunk?

[Ralf Wildenhues]

Hello,

* Xin He wrote on Thu, Jun 23, 2011 at 01:23:53PM CEST:
> make[3]: Entering directory `/home/ehhexxn/git/ompi/ompi/include'
>   FC mpif90-ext.lo
> libtool: compile: unrecognized option `-c'
> libtool: compile: Try `libtool --help' for more information.
> make[3]: *** [mpif90-ext.lo] Error 1

Do a 'make V=1' to see details.

You will most likely find that configure didn't find a suitable Fortran
(FC) compiler for you to use.  Check the config.log of the enclosing
package to verify.

Hope that helps.

Cheers,
Ralf

Re: [OMPI devel] "make check" (libtool) failure on Linux/ppc w/ XLC (1.5rc5 and 1.4.3rc1)

[Ralf Wildenhues]

Hello Paul,

this bug has been addressed in upstream Libtool commit
v2.2.6-59-gb7dbec6:
<http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=v2.2.6-59-gb7dbec6>

It is not wholly fixed yet, but for all cases interesting to OpenMPI.
(A libtoolized package using only Fortran but no C compiler may still
have issues.)

Cheers,
Ralf

* Paul H. Hargrove wrote on Sat, Aug 28, 2010 at 02:39:27AM CEST:
> I reran the build as Ralf requested (setting
> shlibpath_overrides_runpath=yes  in the libtool script).
> The build and check were SUCCESSful!
> 
> I have provided Ralf with the requested files off-list.
> -Paul
> 
> 
> Ralf Wildenhues wrote:
> >* Paul H. Hargrove wrote on Fri, Aug 27, 2010 at 03:54:54AM CEST:
> >>>I am now looking at using IBM's XLC compilers for ILP32 builds on
> >>>the same Linux/PPC64 platform for which I've reported some
> >>>XLC/LP64 bugs.
> >>>
> >>>What I find now is that "make check" is failing with the loader
> >>>unable to find libmpi.so.0.
> >>>This is with both 1.5rc5 and 1.4.3rc1.
> >>>This occurs with xlc, but things are just fine with gcc.
> >
> >>>As I said above, the problem is NOT occuring w/ gcc.  So, I've
> >>>attached the "./libtool --config" output for the xlc and gcc
> >>>builds, which I see differ in more ways than I would have
> >>>expected.
> >
> >The thing that's unexpected to me is the shlibpath_overrides_runpath
> >difference.
> >
> >>While I still don't know the root cause, the following diff between
> >>the libtool-generated wrappers for a gcc and xlc build make the
> >>cause of the "make check" failure fairly obvious:
> >
> >Please set shlibpath_overrides_runpath=yes in the libtool script for
> >xlc, then try 'make clean all check'.  Please send config.log for xlc
> >(packed).
> >
> >Thanks,
> >Ralf

Re: [OMPI devel] Failure (libtool?) to build F90 bindings w/ XLC/PPC64 (1.5rc5 and 1.4.3rc1)

[Ralf Wildenhues]

[ http://www.open-mpi.org/community/lists/devel/2010/08/8398.php ]

Hello Paul,

sorry for the late reply.  This issue you reported is valid; it is
already fixed in upstream Libtool 2.2.8, with commit
v2.2.6-201-g519bf91:


Cheers,
Ralf

* Paul H. Hargrove wrote on Thu, Aug 26, 2010 at 11:12:55PM CEST:
> One of the platforms I've been testing is a Linux/PPC64 (which happens to be
> the front-end to a BG/P, but don't be confused by that - I am NOT trying to
> build for the BG/P).  On the system are IBM's XLC compilers (also sold under
> the ABSoft name).  When passing "-q64" to the xlc compilers to get an LP64
> ABI (default is ILP32), it seems that the scripts for constricting the F90
> bindings somehow end up passing the "-q64" to /usr/bin/ld, which is not
> happy.
> 
> If I don't set {C,CXX,F,FC}FLAGS=-q64 then there is no problem building the
> F90 bindings (for ILP32 ABI).
> 
> If I --disable-mpi-f90 the build is fine (except for the atomic test failure
> from "make check", reported in
> http://www.open-mpi.org/community/lists/devel/2010/08/8369.php)
> 
> Here are the details of the platform:
> 
> $ uname -a
> Linux login1 2.6.16.60-0.67.1-ppc64 #1 SMP Thu Aug 5 10:54:46 UTC 2010
> ppc64 ppc64 ppc64 GNU/Linux
> 
> $ which xlc
> /soft/apps/ibmcmp-aug2010/vac/bg/9.0/bin/xlc
> $ xlc -qversion
> IBM XL C/C++ Advanced Edition for Blue Gene/P, V9.0
> Version: 09.00..0009
[...]

> Here is the configure command:
> 
> $ [path_to]/configure --with-contrib-vt-flags=--with-platform=linux CC=xlc_r
> CXX=xlC_r F77=xlf FC=xlf90 CFLAGS=-q64 CXXFLAGS=-q64 FFLAGS=-q64
> FCFLAGS=-q64
> 
> The problem exists with both 1.5rc5 and 1.4.3rc1.
> 
> Here is the failure from 1.4.3.rc1:
> 
> $ make
> [...]
> make[4]: Entering directory
> `/gpfs/home/hargrove/tmp/openmpi-1.4.3rc1/BLD-xlc_r-64/ompi/mpi/f90'
> /bin/sh ../../../libtool   --mode=link xlf90 -I../../../ompi/include
> -I../../../../ompi/include -I. -I../../../../ompi/mpi/f90
> -I../../../ompi/mpi/f90  -q64 -version-info 0:0:0  -export-dynamic   -o
> libmpi_f90.la -rpath /usr/local/lib mpi.lo mpi_sizeof.lo
> mpi_comm_spawn_multiple_f90.lo mpi_testall_f90.lo mpi_testsome_f90.lo
> mpi_waitall_f90.lo mpi_waitsome_f90.lo mpi_wtick_f90.lo mpi_wtime_f90.lo
> ../../../ompi/libmpi.la -lnsl -lutil
> libtool: link: /usr/bin/ld -m elf64ppc -shared  .libs/mpi.o
> .libs/mpi_sizeof.o .libs/mpi_comm_spawn_multiple_f90.o
> .libs/mpi_testall_f90.o .libs/mpi_testsome_f90.o .libs/mpi_waitall_f90.o
> .libs/mpi_waitsome_f90.o .libs/mpi_wtick_f90.o .libs/mpi_wtime_f90.o
> -L/home/hargrove/tmp/openmpi-1.4.3rc1/BLD-xlc_r-64/orte/.libs
> -L/home/hargrove/tmp/openmpi-1.4.3rc1/BLD-xlc_r-64/opal/.libs
> ../../../ompi/.libs/libmpi.so
> /home/hargrove/tmp/openmpi-1.4.3rc1/BLD-xlc_r-64/orte/.libs/libopen-rte.so
> /home/hargrove/tmp/openmpi-1.4.3rc1/BLD-xlc_r-64/opal/.libs/libopen-pal.so
> -ldl -lnsl -lutil  -q64   -soname libmpi_f90.so.0 -o
> .libs/libmpi_f90.so.0.0.0
> /usr/bin/ld: unrecognized option '-q64'
> /usr/bin/ld: use the --help option for usage information
> make[4]: *** [libmpi_f90.la] Error 1
> make[4]: Leaving directory
> `/gpfs/home/hargrove/tmp/openmpi-1.4.3rc1/BLD-xlc_r-64/ompi/mpi/f90'
> make[3]: *** [all-recursive] Error 1
[...]

> Here is the failure from 1.5rc5 (including re-run w/ V=1)

> $ make V=1
> [...]
> make[4]: Entering directory
> `/gpfs/home/hargrove/tmp/openmpi-1.5rc5/BLD-xlc_r-64/ompi/mpi/f90'
> /bin/sh ../../../libtool  --tag=FC   --mode=link xlf90
> -I../../../ompi/include -I../../../../ompi/include -I.
> -I../../../../ompi/mpi/f90 -I../../../ompi/mpi/f90  -q64 -version-info 0:0:0
> -export-dynamic  -o libmpi_f90.la -rpath /usr/local/lib mpi.lo mpi_sizeof.lo
> mpi_comm_spawn_multiple_f90.lo mpi_testall_f90.lo mpi_testsome_f90.lo
> mpi_waitall_f90.lo mpi_waitsome_f90.lo mpi_wtick_f90.lo mpi_wtime_f90.lo
> ../../../ompi/mpi/f77/libmpi_f77.la -lnsl  -lutil
> libtool: link: /usr/bin/ld -m elf64ppc -shared  .libs/mpi.o
> .libs/mpi_sizeof.o .libs/mpi_comm_spawn_multiple_f90.o
> .libs/mpi_testall_f90.o .libs/mpi_testsome_f90.o .libs/mpi_waitall_f90.o
> .libs/mpi_waitsome_f90.o .libs/mpi_wtick_f90.o .libs/mpi_wtime_f90.o
> -L/home/hargrove/tmp/openmpi-1.5rc5/BLD-xlc_r-64/ompi/.libs
> ../../../ompi/mpi/f77/.libs/libmpi_f77.so
> /home/hargrove/tmp/openmpi-1.5rc5/BLD-xlc_r-64/ompi/.libs/libmpi.so -ldl
> -lnsl -lutil  -q64   -soname libmpi_f90.so.0 -o .libs/libmpi_f90.so.0.0.0
> /usr/bin/ld: unrecognized option '-q64'
> /usr/bin/ld: use the --help option for usage information
> make[4]: *** [libmpi_f90.la] Error 1
> make[4]: Leaving directory
> `/gpfs/home/hargrove/tmp/openmpi-1.5rc5/BLD-xlc_r-64/ompi/mpi/f90'
[...]


> Based on the make output this might be a libtool issue.
> 
> However, I noticed the following comment in ompi/mpi/f90/Makefile.am:
> # This Makefile.am is quite complex and confusing.  Part of the
> # problem is that Libtool (v1.5.18) does not understand F90, so we
> # have to do a 

Re: [OMPI devel] update configury for Autoconf 2.68

[Ralf Wildenhues]

* Jeff Squyres wrote on Fri, Sep 24, 2010 at 03:26:46PM CEST:
> On Sep 23, 2010, at 2:15 PM, Ralf Wildenhues wrote:
> >> Is the silent-rules clause in AM_INIT_AUTOMAKE exactly equivalent to
> >> calling AM_SILENT_RULES?
> > 
> > Yes.
> 
> Weird -- when I do this:

> -AM_INIT_AUTOMAKE([foreign dist-bzip2 subdir-objects no-define 1.10 
> tar-ustar])
> +AM_INIT_AUTOMAKE([foreign dist-bzip2 subdir-objects no-define 1.11 
> silent-rules tar-ustar])
>  
> -# If Automake supports silent rules, enable them.
> -m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
> -

> I do not get silent rules behavior -- I still get the old verbose behavior.  
> This is with:

Ah, yes, you are exactly right of course, and I was hurrying and reading
the code wrong.  With the 'silent-rules' option, the user still needs to
pass --enable-silent-rules to configure.  IOW, the default is different.

Cheers,
Ralf

Re: [OMPI devel] Setting AUTOMAKE_JOBS

[Ralf Wildenhues]

Hello Ralph,

wow, that's not good to hear.  I knew the perl ithreads implementation
wasn't all that efficient, but causing a deadlock sounds like you have
more trouble than just perl; at least I hope so.  For reference, can
you send 'perl -V' output (if you like, to the bug-automake at gnu.org
list).

Thanks,
Ralf

* Ralph Castain wrote on Fri, Sep 24, 2010 at 03:12:16AM CEST:
> I found one major negative to this change - it assumes that my build is
> being done in exclusion of anything else on my computer. Unfortunately, this
> is never true.
> 
> So my laptop hemorrhaged itself into frozen silence, overheated to the point
> of being burning hot, and had to have its battery yanked to stop the runaway
> behavior. Not a really good thing.
> 
> I would suggest you default this "heuristic" out, and let someone set it to
> use multiple runs if-and-only-if they want it. Hate to cite the lowest
> common denominator, but this was a very nasty surprise.
> 
> 
> 
> On Wed, Sep 22, 2010 at 7:50 AM, Jeff Squyres  wrote:
> 
> > Some of you may be unaware that recent versions of automake can run in
> > parallel.  That is, automake will run in parallel with a degree of (at most)
> > $AUTOMAKE_JOBS.  This can speed up the execution time of autogen.pl quite
> > a bit on some platforms.  On my cluster at cisco, here's a few quick timings
> > of the entire autogen.pl process (of which, automake is the bottleneck):
> >
> > $AUTOMAKE_JOBS   Total wall time
> > valueof autogen.pl
> > 83:01.46
> > 42:55.57
> > 23:28.09
> > 14:38.44
> >
> > This is an older Xeon machine with 2 sockets, each with 2 cores.
> >
> > There's a nice performance jump from 1 to 2, and a smaller jump from 2 to
> > 4.  4 and 8 are close enough to not matter.  YMMV.
> >
> > I just committed a heuristic to autogen.pl to setenv AUTOMAKE_JOBS if it
> > is not already set (https://svn.open-mpi.org/trac/ompi/changeset/23788):
> >
> > - If lstopo is found in your $PATH, runs it and count how many PU's
> > (processing units) you have.  It'll set AUTOMAKE_JOBS to that number, or a
> > maximum of 4 (which is admittedly a further heuristic).
> > - If lstopo is not found, it just sets AUTOMAKE_JOBS to 2.
> >
> > Enjoy.
> >
> > --
> > Jeff Squyres
> > jsquy...@cisco.com
> > For corporate legal information go to:
> > http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] update configury for Autoconf 2.68

[Ralf Wildenhues]

* Jeff Squyres wrote on Thu, Sep 23, 2010 at 08:11:41PM CEST:
> On Sep 23, 2010, at 2:03 PM, Ralf Wildenhues wrote:
> 
> >> Is the call to AM_SILENT_RULES now moot because it's listed in 
> >> AM_INIT_AUTOMAKE?
> > 
> > Oh, just drop that hunk, that was leftover junk in my tree and isn't
> > really needed any more; I didn't mean to send it along.
> 
> Is the silent-rules clause in AM_INIT_AUTOMAKE exactly equivalent to
> calling AM_SILENT_RULES?

Yes.

(The macro differs in that it also accepts an undocumented optional
argument to specify the default configure-time setting, but if you find
out and use it, some distributor people might get a bit annoyed ...)

>  We already require AM 1.11.1 or higher -- so
> it might be a moot point to check to see if AM_SILENT_RULES is defined
> or not...

Indeed.

Cheers,
Ralf

Re: [OMPI devel] update configury for Autoconf 2.68

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Thu, Sep 23, 2010 at 03:31:32PM CEST:
> One very minor question: I notice you added silent-rules to
> AM_INIT_AUTOMAKE (and bumped the required version, too), but still
> left in the call to AM_SILENT_RULES:
> 
> -AM_INIT_AUTOMAKE([foreign dist-bzip2 subdir-objects no-define 1.10 
> tar-ustar])
> +AM_INIT_AUTOMAKE([foreign dist-bzip2 subdir-objects no-define 1.10c 
> silent-rules tar-ustar])
> 
> # If Automake supports silent rules, enable them.
> m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
> 
> Is the call to AM_SILENT_RULES now moot because it's listed in 
> AM_INIT_AUTOMAKE?

Oh, just drop that hunk, that was leftover junk in my tree and isn't
really needed any more; I didn't mean to send it along.

Thanks,
Ralf

[OMPI devel] update configury for Autoconf 2.68

[Ralf Wildenhues]

Hello OpenMPI developers,

just-released Autoconf 2.68 contains more stringent checks and warnings
about stuff passed to AC_{COMPILE,LINK,RUN}_IFELSE and AC_TRY_* macros.
Specifically, it will warn if C or C++ programs are not generated with
AC_LANG_SOURCE or AC_LANG_PROGRAM, in order to avoid accidentally
omitting needed headers, or mess up arguments in some way.  This has
happened a few times to Autoconf users.

These checks make more stringent assumptions on correct m4 quoting of
macros; IOW, there are mis-quotings which will cause false warnings to
be emitted.

In cases where generating sources without AC_LANG_SOURCE or
AC_LANG_PROGRAM is done intentionally, the macro
AC_LANG_DEFINES_PROVIDED should be expanded inside the source.

The patch below silences all warnings.  I've used
AC_LANG_DEFINES_PROVIDED a couple of times in
config/ompi_check_vendor.m4, assuming that the tests are really meant
to not include any headers.  In a couple of other cases, esp in
opal/mca/if/windows/configure.m4 and
opal/mca/installdirs/windows/configure.m4, I've added the default
include headers, assuming that they should not hurt.

I've tested the patch with Autoconf 2.68 as well as Autoconf 2.67 on
GNU/Linux, but done no other testing.

For complete absence of warnings from Autoconf 2.68 you will also need
to update the Libtool macros from upcoming 2.4.0.

Cheers,
Ralf

Index: opal/mca/installdirs/windows/configure.m4
===
--- opal/mca/installdirs/windows/configure.m4   (revision 23790)
+++ opal/mca/installdirs/windows/configure.m4   (working copy)
@@ -27,18 +27,20 @@
 # registry. We should first check that the function is defined,
 # and then check for it's presence in the kernel32 library.
 AC_MSG_CHECKING(for working RegOpenKeyEx)
-AC_TRY_RUN( [#include 
+AC_RUN_IFELSE([AC_LANG_PROGRAM([AC_INCLUDES_DEFAULT
+#include ], [
 int main( int argc, char** argv ) {
 RegOpenKeyEx( HKEY_CURRENT_USER, "Software\\Open MPI", 0, KEY_READ, NULL);
-return 0; }],
+return 0; }])],
 [AC_MSG_RESULT([yes])
  $1],
 [AC_MSG_RESULT([no])
  $2],
-[AC_COMPILE_IFELSE([#include 
+[AC_COMPILE_IFELSE([AC_LANG_PROGRAM([AC_INCLUDES_DEFAULT
+#include ], [
 int main( int argc, char** argv ) {
 RegOpenKeyEx( HKEY_CURRENT_USER, "Software\\Open MPI", 0, KEY_READ, NULL);
-return 0; }],
+return 0; }])],
 [AC_MSG_RESULT([yes])
  $1],
 [AC_MSG_RESULT([no])
Index: opal/mca/paffinity/hwloc/hwloc/config/hwloc.m4
===
--- opal/mca/paffinity/hwloc/hwloc/config/hwloc.m4  (revision 23790)
+++ opal/mca/paffinity/hwloc/hwloc/config/hwloc.m4  (working copy)
@@ -430,13 +430,13 @@

 _HWLOC_CHECK_DECL([sched_setaffinity], [
   AC_MSG_CHECKING([for old prototype of sched_setaffinity])
-  AC_COMPILE_IFELSE(
+  AC_COMPILE_IFELSE([
 AC_LANG_PROGRAM([[
   #define _GNU_SOURCE
   #include 
   static unsigned long mask;
   ]], [[ sched_setaffinity(0, (void*) );
-  ]]),
+  ]])],
 AC_DEFINE([HWLOC_HAVE_OLD_SCHED_SETAFFINITY], [1], [Define to 1 if 
glibc provides the old prototype of sched_setaffinity()])
 AC_MSG_RESULT([yes]),
 AC_MSG_RESULT([no])
@@ -447,19 +447,19 @@
 ]])

 AC_MSG_CHECKING([for working CPU_SET])
-AC_LINK_IFELSE(
+AC_LINK_IFELSE([
   AC_LANG_PROGRAM([[
 #include 
 cpu_set_t set;
 ]], [[ CPU_ZERO(); CPU_SET(0, );
-]]),
+]])],
 AC_DEFINE([HWLOC_HAVE_CPU_SET], [1], [Define to 1 if the CPU_SET macro 
works])
 AC_MSG_RESULT([yes]),
 AC_MSG_RESULT([no])
 )

 AC_MSG_CHECKING([for working CPU_SET_S])
-AC_LINK_IFELSE(
+AC_LINK_IFELSE([
   AC_LANG_PROGRAM([[
   #include 
   cpu_set_t *set;
@@ -468,7 +468,7 @@
   CPU_ZERO_S(CPU_ALLOC_SIZE(1024), set);
   CPU_SET_S(CPU_ALLOC_SIZE(1024), 0, set);
   CPU_FREE(set);
-]]),
+]])],
 AC_DEFINE([HWLOC_HAVE_CPU_SET_S], [1], [Define to 1 if the CPU_SET_S 
macro works])
 AC_MSG_RESULT([yes]),
 AC_MSG_RESULT([no])
@@ -570,7 +570,7 @@
 AC_MSG_CHECKING([for cpuid])
 old_CPPFLAGS="$CPPFLAGS"
 CFLAGS="$CFLAGS -I$HWLOC_top_srcdir/include"
-AC_COMPILE_IFELSE(AC_LANG_PROGRAM([[
+AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
 #include 
 #include 
   ]], [[
@@ -580,7 +580,7 @@
   printf("highest cpuid %x\n", eax);
   return 0;
 }
-  ]]), [
+  ]])], [
   AC_MSG_RESULT([yes])
   AC_DEFINE(HWLOC_HAVE_CPUID, 1, [Define to 1 if you have cpuid])
   hwloc_have_cpuid=yes
Index: opal/mca/if/windows/configure.m4
===
--- opal/mca/if/windows/configure.m4(revision 23790)
+++ 

Re: [OMPI devel] Setting AUTOMAKE_JOBS

[Ralf Wildenhues]

Hi Jeff,

adding bug-automake in Cc: (non-subscribers can't post to the Open MPI
list, so please remove that Cc: in case)

* Jeff Squyres wrote on Wed, Sep 22, 2010 at 03:50:19PM CEST:
> $AUTOMAKE_JOBS   Total wall time
> valueof autogen.pl
> 83:01.46
> 42:55.57
> 23:28.09
> 14:38.44
> 
> This is an older Xeon machine with 2 sockets, each with 2 cores.

Thanks for the measurements!  I'm a bit surprised that the speedup is
not higher.  Do you have timings as to how much of the autogen.pl time
is spent inside automake?

IIRC the pure automake part for OpenMPI would speed up better on bigger
systems, my old numbers from two years ago are here:
http://lists.gnu.org/archive/html/automake-patches/2008-10/msg00055.html

Cheers,
Ralf

Re: [OMPI devel] AS_VAR_COPY failure (Re: === CREATE FAILURE (v1.4) ===)

[Ralf Wildenhues]

Hi Jeff,

Yes, sorry for not making this clear earlier: AS_VAR_COPY is fairly new,
and was introduced because AS_VAR_GET has problems.  You should be able
to workaround it by placing something like

  m4_ifndef([AS_VAR_COPY],
[m4_define([AS_VAR_COPY],
   [AS_LITERAL_IF([$1[]$2], [$1=$$2], [eval $1=\$$2])])])

early into the m4 stream (untested).

Cheers,
Ralf

* Jeff Squyres wrote on Thu, Sep 09, 2010 at 07:34:07AM CEST:
> I'm checking into this.
> 
> I'm guessing that the problem is that the autoconf we use to build the 1.4 
> tarballs doesn't have AS_VAR_COPY.  Arrgghhh...
> 
> 
> 
> On Sep 9, 2010, at 3:02 AM, MPI Team wrote:
> 
> > 
> > ERROR: Command returned a non-zero exist status (v1.4):
> >   ./autogen.sh
> > 
> > Start time: Wed Sep  8 21:00:07 EDT 2010
> > End time:   Wed Sep  8 21:02:49 EDT 2010
> > 
> > ===
> > [... previous lines snipped ...]
> > 
> > *** Found configure.(in|ac)
> > ***   
> > /home/mpiteam/openmpi/nightly-tarball-build-root/v1.4/create-r23733/ompi/ompi/contrib/vt/vt/extlib/otf
> > 
> > *** Running GNU tools
> > [Running] libtoolize --automake --copy
> > [Running] aclocal
> > [Running] autoheader
> > [Running] autoconf
> > [Running] automake --foreign -a --copy --include-deps
> >  ++ Patching configure for Libtool PGI 10 fortran compiler name
> >  ++ Patching configure for Libtool PGI version number regexps
> > 
> > <== Back in 
> > /home/mpiteam/openmpi/nightly-tarball-build-root/v1.4/create-r23733/ompi/ompi/contrib/vt
> > <== autogen.sh continuing...
> > 
> > *** Found configure.(in|ac)
> > ***   
> > /home/mpiteam/openmpi/nightly-tarball-build-root/v1.4/create-r23733/ompi
> > 
> > *** Running GNU tools
> > [Running] autom4te --language=m4sh ompi_get_version.m4sh -o 
> > ompi_get_version.sh
> > [Running] libtoolize --automake --copy --ltdl
> > ** Adjusting libltdl for OMPI :-(
> >   ++ patching for argz bugfix in libtool 1.5
> >  -- your libtool doesn't need this! yay!
> >   ++ patching 64-bit OS X bug in ltmain.sh
> >  -- your libtool doesn't need this! yay!
> >   ++ RTLD_GLOBAL in libltdl
> >  -- your libltdl doesn't need this! yay!
> > grep: configure: No such file or directory
> >   ++ patching for ifort (LT 2.2.0-4)
> > -- your libltdl doesn't need this! yay!
> > [Running] aclocal -I config
> > [Running] autoheader
> > ** Adjusting libtool for OMPI :-(
> >   ++ patching for pathscale multi-line output (LT 2.x)
> >   ++ patching for ifort (LT 2.2.0-4)
> > -- your libltdl doesn't need this! yay!
> >   ++ patching to remove solaris Cstd
> >   ++ patching for Sun Studio Fortran compilers
> > [Running] autoconf
> > configure:36430: error: possibly undefined macro: AS_VAR_COPY
> >  If this token and others are legitimate, please use m4_pattern_allow.
> >  See the Autoconf documentation.
> > 
> > -
> > It seems that the execution of "autoconf" has failed.  See above for
> > the specific error message that caused it to abort.
> > -
> > 
> > ===
> > 
> > Your friendly daemon,
> > Cyrador
> > ___
> > testing mailing list
> > test...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/testing
-- 
Ralf Wildenhues  Institut fuer Numerische Simulation, Universitaet Bonn
 Wegelerstrasse 6, D-53115 Bonn
tel: +49 228 733178  mailto:wildenh...@ins.uni-bonn.de
fax: +49 228 737527  http://wissrech.ins.uni-bonn.de/people/wildenhues.html

Re: [OMPI devel] "make check" (libtool) failure on Linux/ppc w/ XLC (1.5rc5 and 1.4.3rc1)

[Ralf Wildenhues]

* Paul H. Hargrove wrote on Fri, Aug 27, 2010 at 03:54:54AM CEST:
> >I am now looking at using IBM's XLC compilers for ILP32 builds on
> >the same Linux/PPC64 platform for which I've reported some
> >XLC/LP64 bugs.
> >
> >What I find now is that "make check" is failing with the loader
> >unable to find libmpi.so.0.
> >This is with both 1.5rc5 and 1.4.3rc1.
> >This occurs with xlc, but things are just fine with gcc.

> >As I said above, the problem is NOT occuring w/ gcc.  So, I've
> >attached the "./libtool --config" output for the xlc and gcc
> >builds, which I see differ in more ways than I would have
> >expected.

The thing that's unexpected to me is the shlibpath_overrides_runpath
difference.

> While I still don't know the root cause, the following diff between
> the libtool-generated wrappers for a gcc and xlc build make the
> cause of the "make check" failure fairly obvious:

Please set shlibpath_overrides_runpath=yes in the libtool script for
xlc, then try 'make clean all check'.  Please send config.log for xlc
(packed).

Thanks,
Ralf

Re: [OMPI devel] "make check" (libtool?) failure on Solaris/SPARC (1.5rc5 and 1.4.3rc1)

[Ralf Wildenhues]

* Paul H. Hargrove wrote on Thu, Aug 26, 2010 at 10:14:22PM CEST:
>  I just had a thought on this one:  In my environment on this
> platform I have LD_LIBRARY_PATH_32 and LD_LIBRARY_PATH_64 set.  It
> seems possible to me that this is causing the loader to ignore the
> LD_LIBRARY_PATH setting that libtool's wrapper script sets when
> executing uninstalled programs in the build directory, as during
> "make check".
>  So, I tried removing these from my environment and editing
> ~/.bashrc not to set them.  The result is a SUCCESSful "make check"!

Cool.  We need to fix that in Libtool then.

Thanks for tracking this down!
Ralf

Re: [OMPI devel] "make check" (libtool?) failure on Solaris/SPARC (1.5rc5 and 1.4.3rc1)

[Ralf Wildenhues]

Hello Paul,

* Paul H. Hargrove wrote on Thu, Aug 26, 2010 at 01:58:11AM CEST:
> I have been able to configure and build both 1.5rc5 and 1.4.3rc1 on
> Solaris 10 for SPARC, using Sun C 5.10.
> I have also build 1.5rc5 w/ gcc-3.3.2 (and expect 1.4.3rc1 to build
> w/ gcc as well, once I have time)
> 
> All 3 builds fail "make check" in a way that suggests to me that
> libtool is not working correctly on this platform.

Can you send output of './libtool --config' for the platforms?

Thanks,
Ralf

Re: [OMPI devel] Fixes to OpenMPI-1.4.2 for PGI compilers

[Ralf Wildenhues]

Hello Larry, Jeff,

I'm one of the Libtool developers.  I'm happy to take things over to the
Libtool source tree and help get them incorporated.

Jeff, any chance you could give me a heads-up when all of them are
incorporated in one of the OpenMPI branches so I can pick them up from
there?

Thanks,
Ralf

* Larry Baker wrote on Fri, Aug 20, 2010 at 07:15:05PM CEST:
> Without the 1.4.2 fix in configure to correctly parse the PGI
> version no. (the same fix you have in 1.5rc5), I'm pretty sure the
> make of 1.4.2 fails for PGI 10.x.  Also, I think the inline assembly
> of atomic instructions might not be correct for the C++ version of
> the library without the other fix in configure to hard-code
> disabling that feature for PGI.
> 
> I don't know anything about Libtool, like where it comes from, which
> pieces of OpenMPI come from Libtool, and, probably more significant,
> how those pieces might have been modified when they were
> incorporated into OpenMPI.  The fixes to configure and libtool.m4
> that I sent seem to already be well known.  (They were passed on to
> me when I fixed them while installing netCDF.) I'm happy to report
> bugs and fixes to the developers of the code I use.  I really have
> to leave it up to them to decide whether to pass them on to their
> upstream providers.
> 
> Larry Baker
> US Geological Survey
> 650-329-5608
> ba...@usgs.gov
> 
> On Aug 20, 2010, at 8:24 AM, Jeff Squyres wrote:
> 
> >On Aug 17, 2010, at 9:50 PM, Ralf Wildenhues wrote:
> >
> >>>I patched OpenMPI 1.4.2 to fix some problems/warnings when using the
> >>>PGI compilers.
> >>
> >>FWIW, if there are fixes suitable for upstream Libtool in your
> >>changes,
> >>we would be glad if you could send them as bug reports or even
> >>'diff -u'
> >>patches to the bug-libtool at gnu.org list.  Line numbers don't help
> >>because they change too quickly, and issues might have been
> >>fixed in the
> >>git version of Libtool already.
> >
> >Yes, that would be great -- would you mind submitting these
> >upstream to Libtool?  We try to do that when I fix something so
> >that it's there for us without patching in future releases.
> >
> >Thanks for the v1.4 fixes!  But given that you also submitted them
> >for v1.5, since none of them are drastic correctness issues, we
> >might well opt to only fix them in the v1.5 series.

Re: [OMPI devel] v1.5 .so version numbers

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Thu, Jun 03, 2010 at 09:34:16PM CEST:
> SHORT VERSION: We broke ABI from the 1.4 series to the v1.5 series.  I
> propose changing all the libtool .so version numbers as shown below to
> enforce that break.  Can someone sanity check this?

Looks sane to me, with the details you have given.

Cheers,
Ralf

Re: [OMPI devel] The" Missing Symbol" issue and OpenMPI on NetBSD

[Ralf Wildenhues]

Hello Kevin,

* kevin.buck...@ecs.vuw.ac.nz wrote on Tue, May 11, 2010 at 06:42:01AM CEST:
> That is a file that gets patched in the NetBSD build as follows
> 
> $diff opal/mca/base/mca_base_component_find.c{.orig,}
> 44,46d43
> <   #ifndef __WINDOWS__
> < #include "opal/libltdl/ltdl.h"
> <   #else
> 48d44
> <   #endif
> 
> ie we have taken out the inclusion of
> 
> opal/libltdl/ltdl.h
> 
> to force the use of the NetBSD "ltdl.h" one, which I guess might point
> to something underlying the issue but as to what ...

Which libltdl version is that NetBSD ltdl.h from?  Which version is in
opal/libltdl?  Have you tried not doing the above change?

libltdl 2.2.x has incompatible changes over 1.5.x, both in the library
as well as in the header, as well as (I think) in preloaded modules.

Cheers,
Ralf

Re: [OMPI devel] bug with /bin/sh and /bin/ksh

[Ralf Wildenhues]

Hello,

* Jeff Squyres wrote on Mon, May 03, 2010 at 04:04:59PM CEST:
> > On Apr 29, 2010, at 11:24 AM, Jonathan Vincent wrote:
> > 
> > > sh -c '/usr/bin/env FOO=bar (echo hello)'
> > > ksh -c '/usr/bin/env FOO=bar (echo hello)'
> > >
> > > is not valid
> > > sh -c '/usr/bin/env FOO=bar echo hello'
> > > works.
> > 
> > Per your later mails, plm_rsh_module.c does this:
> > 
> > tmp = opal_argv_split("( test ! -r ./.profile || . ./.profile;", ' 
> > ');
> > 
> > (and later adds the closing ")").  You reported the final line to be:
> > 
> > /usr/bin/env 
> > LD_LIBRARY_PATH=/pdc/vol/openmpi/1.4.1/intel/lib:/pdc/vol/icompilers/11.1/icc/lib/intel64:/pdc/vol/i-compilers/11.1/ifort/lib/intel64
> >  ( test ! -r ./.profile || . ./.profile; 
> > /pdc/vol/openmpi/1.4.1/intel/bin/orted -mca ess env -mca orte_ess_jobid 
> > 284360704 -mca orte_ess_vpid 3 -mca orte_ess_num_procs 5 --hnp-uri 
> > "284360704.0;tcp://193.11.170.208:49530" )
> > 
> > What would the correct syntax be?

Well, after the arguments setting the variables, portable env really
only takes a command line with arguments, not something that is passed
to the shell again.  You could work around this by constructing
something like

  /usr/bin/env var=val ... sh -c 'script ...'

>> With the || and ;, are the () really unnecessary?

Well, neither the () nor the || nor ; really belong in a command-line
that is not interpreted by a shell.

Cheers,
Ralf

Re: [OMPI devel] Porting OpenMPI to a new system

[Ralf Wildenhues]

Hello Ioannis,

* Ioannis E. Venetis wrote on Wed, Apr 28, 2010 at 05:34:47PM CEST:
>   b) The cross-compilation environment should be run on a Linux x86_64
> system. The cross-compiler, libraries, etc are already working, which
> means that only OpenMPI needs to be ported right now. We use this
> environment together with a simulator of the system, in order to
> evaluate any changes in the architecture of the system. Using the
> typical configure options of most applications, we would like to have
> something like the following options in this case when building OpenMPI:
> 
> --host=x86_64-linux-gnu --build=

It looks like you are mixing up host and build.  build is the system you
run configure on, host is the system the generated OpenMPI will run on.
Also, x86_64-linux-gnu is not a valid triplet, as it is missing the
vendor bit.  x86_64-unknown-linux-gnu is valid.

If your new system is sufficiently different from existing systems, then
all kinds of files from the build system may need updating: config.guess
and config.sub (email to send patches to is listed in the files), the
Libtool macro files (write to bug-libt...@gnu.org).  But if you already
have a working cross compiler it doesn't seem to me like these steps
would be necessary.

For proper OpenMPI code I can't tell you which parts would need
adjustment.

Cheers,
Ralf

Re: [OMPI devel] 答复: 答复: Migrate the OpenMPI to VxWorks

[Ralf Wildenhues]

* 张晶 wrote on Fri, Apr 16, 2010 at 04:50:21PM CEST:
> I think I should switch the host now in the windows to the linux ,or I will
> have little chance to build the autotool, Thank you for your advice !

autotools work just fine under Windows.  For example, Cygwin or MinGW
ship pre-built autotools versions.

Cheers,
Ralf

Re: [OMPI devel] Migrate the OpenMPI to VxWorks

[Ralf Wildenhues]

* Ralph Castain wrote on Fri, Apr 16, 2010 at 05:35:37AM CEST:
> I have not personally tried, but I am pretty sure that you can install
> the autotools under VxWorks - have you tried to download the latest
> autotool tarballs and build them?

I don't know if that works well out of the box, but if you build any of
the autotools, run their testsuites and find any failures, then we would
be happy to hear about them at the respective mailing lists.  If the
testsuites pass, you can be fairly confident that they work well on your
system.

To find out where the OpenMPI configure script hangs, try running it
after adding 'set -x' as second line to the script.  The output will be
large, so beware.  If /bin/sh is not a Posix shell, you might want to
try something like
  CONFIG_SHELL=/bin/bash; export CONFIG_SHELL
  $CONFIG_SHELL ./configure [OPTIONS...]

instead.

Hope that helps,
Ralf

Re: [OMPI devel] RFC: ABI break between 1.4 and 1.5 / .so versioning

[Ralf Wildenhues]

* Jeff Squyres wrote on Wed, Feb 17, 2010 at 11:51:42PM CET:
> On Feb 17, 2010, at 3:05 PM, Ralf Wildenhues wrote:
> 
> > > The issue is that if the user has to specify -static to their linker,
> > > they *also* have to specify --ompi:static, or Bad Things will happen.
> > > Or, if they don't specify -static but *only* specify --ompi:static,
> > > Bad Things will happen.  In short: it seems like adding yet another
> > > wrapper-compiler-specific flag to the MPI ecosystem will cause
> > > confusion, fear, and possibly the death of some cats.
> > 
> > Do you care for omitting -lopen-pal and -lorte only for capable Linux
> > systems?  With new-enough binutils, you should be able to use
> > -Wl,--as-needed -Wl,--no-as-needed around these two libs.
> 
> Mmmm.  Good point.  But I don't think it helps us on Solaris or OS X, does 
> it?  (maybe it does on OS X?)  Or do all linkers have some kind of option 
> like this?  (this *might* be a way out, but I would probably need to be 
> convinced :-) )

No, I think only binutils ld (and gold) have this.  Sorry.

> > I'm not entirely sure I understand your argumentation for why libmpi
> > from 1.5.x has to be binary incompatible, but I haven't fully thought
> > through this yet.
> 
> The context for this issue is so long that much was left out of my mail.  
> Here's this particular issue in a nutshell:
> 
> - Open MPI v1.4.1 has libmpi at 0:1:0 and libopen-rte and libopen-pal both at 
> 0:0:0.
> - Open MPI v1.4.1 links MPI apps against -lmpi -lopen-rte -lopen-pal.
> - If we start .so versioning properly in v1.5, it's likely that libopen-rte 
> and libopen-pal will both be 1:0:0.
>   --> Note that these are both internal libraries; there are no symbols in 
> these libraries that are used in the MPI applications.
> - Open MPI v1.5 libmpi *could* be 1:0:1.
> - Hence, an a.out created for OMPI v1.4.1 would work fine with v1.5 libmpi.
> - But that a.out would not work with v1.5 libopen-rte and libopen-pal.

You could probably create fake empty libopen-rte and libopen-pal stub
libraries with 0:0:0 purely for the sake of allowing such an a.out to
still work (on systems with versioned sonames[1]).  Since this doesn't
actually use any of the APIs from those libraries, there is no problem
here, and your 1.5 libmpi will pull in the 1:0:0 versions of the other
two libraries.

I understand if you decide not to go such ways, and in that case, I
agree that bumping libmpi to 1:0:0 won't cause much extra pain.

Cheers,
Ralf

[1] This includes many but probably not all systems with shared
libraries.  E.g., I think AIX without runtimelinking (-Wl,-brtl)
would have a problem.

Re: [OMPI devel] configure question

[Ralf Wildenhues]

Hello Greg,

* Greg Watson wrote on Tue, Feb 16, 2010 at 07:03:30PM CET:
> When I run configure under Snow Leopard (this is OMPI 1.3.4), I get the 
> following:
> 
> checking if C and Fortran 77 are link compatible... no
> **
> It appears that your Fortran 77 compiler is unable to link against
> object files created by your C compiler.  This typically indicates
> one of a few possibilities:
> 
>   - A conflict between CFLAGS and FFLAGS
>   - A problem with your compiler installation(s)
>   - Different default build options between compilers (e.g., C
> building for 32 bit and Fortran building for 64 bit)
>   - Incompatible compilers
> 
> Such problems can usually be solved by picking compatible compilers
> and/or CFLAGS and FFLAGS.  More information (including exactly what
> command was given to the compilers and what error resulted when the
> commands were executed) is available in the config.log file in this
> directory.
> **
> configure: error: C and Fortran 77 compilers are not link compatible.  Can 
> not continue.
> 
> Anyone know of the top of their head what these options would be, or even if 
> it is possible?

Well, did you take a look at the corresponding bits in the config.log
file?  Can you post them?

Thanks,
Ralf

Re: [OMPI devel] RFC: Adding -DOPEN_MPI=1 to mpif77 and mpif90

[Ralf Wildenhues]

* Jeff Squyres wrote on Thu, Feb 11, 2010 at 12:41:15PM CET:
> On Feb 11, 2010, at 1:00 AM, Ralf Wildenhues wrote:
> > * Jeff Squyres wrote on Wed, Feb 10, 2010 at 10:02:27PM CET:
> > > WHAT: Add -DOPEN_MPI=1 to the mpif77 and mpif90 command lines
> > 
> > It won't work with IBM xlf which needs -WF,-D.  I'm sure there are other
> > Fortran compilers that don't grok -D either (and may not have any other
> > flag), but I'm not sure whether OpenMPI cares about them.
> 
> Ah, good!  
> 
> If we care, it is easy enough to add a configure test to figure this
> kind of stuff out.  

Sure.

> Are you aware of any other Fortran compilers that don't accept -D, and
> if so, what flags they *do* accept?

I've come across old Fortran 77 compilers that don't do any preprocessing
and thus don't accept any related flags.  I don't remember details
though, and I'm not aware of any currently-used ones; my experience with
Fortran compilers is limited though.

Cheers,
Ralf

Re: [OMPI devel] RFC: Adding -DOPEN_MPI=1 to mpif77 and mpif90

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Wed, Feb 10, 2010 at 10:02:27PM CET:
> WHAT: Add -DOPEN_MPI=1 to the mpif77 and mpif90 command lines

> But we can put -DOPEN_MPI=1 in the argv that the wrapper adds.  This
> seems like a safe way to add it; it makes no difference whether the
> Fortran file is set to the preprocessor or not when it is compiled.

It won't work with IBM xlf which needs -WF,-D.  I'm sure there are other
Fortran compilers that don't grok -D either (and may not have any other
flag), but I'm not sure whether OpenMPI cares about them.

Cheers,
Ralf

Re: [OMPI devel] Dynamic languages, dlopen() issues, and symbol visibility of libtool ltdl API in current trunk

[Ralf Wildenhues]

As a workaround, Lisandro could just pre-seed the cache variables of the
respective configure tests that come out wrong.

  ./configure lt_cv_dlopen_self=yes lt_cv_dlopen_self_static=yes

HTH.

Cheers,
Ralf

* Jeff Squyres wrote on Mon, Sep 21, 2009 at 02:45:28PM CEST:
> Ick; I appreciate Lisandro's quandry, but don't quite know what to do.
> 
> How about keeping libltdl fvisibility=hidden inside mpi4py?
> 
> 
> On Sep 17, 2009, at 11:16 AM, Josh Hursey wrote:
> 
> >So I started down this road a couple months ago. I was using the
> >lt_dlopen() and friends in the OPAL CRS self module. The visibility
> >changes broke that functionality. The one solution that I started
> >implementing was precisely what you suggested, wrapping a subset the
> >libtool calls and prefixing them with opal_*. The email thread is
> >below:
> >   http://www.open-mpi.org/community/lists/devel/2009/07/6531.php
> >
> >The problem that I hit was that libtool's build system did not play
> >well with the visibility symbols. This caused dlopen to be disabled
> >incorrectly. The libtool folks have a patch and, I believe, they are
> >planning on incorporating in the next release. The email thread is
> >below:
> >   http://thread.gmane.org/gmane.comp.gnu.libtool.patches/9446
> >
> >So we would (others can speak up if not) certainly consider such a
> >wrapper, but I think we need to wait for the next libtool release
> >(unless there is other magic we can do) before it would be usable.
> >
> >Do others have any other ideas on how we might get around this in the
> >mean time?
> >
> >-- Josh
> >
> >
> >On Sep 16, 2009, at 5:59 PM, Lisandro Dalcin wrote:
> >
> >> Hi all.. I have to contact you again about the issues related to
> >> dlopen()ing libmpi with RTLD_LOCAL, as many dynamic languages
> >(Python
> >> in my case) do.
> >>
> >> So far, I've been able to manage the issues (despite the "do
> >nothing"
> >> policy from Open MPI devs, which I understand) in a more or less
> >> portable manner by taking advantage of the availability of libtool
> >> ltdl symbols in the Open MPI libraries (specifically, in
> >libopen-pal).
> >> For reference, all this hackery is here:
> >> http://code.google.com/p/mpi4py/source/browse/trunk/src/compat/openmpi.h
> >>
> >> However, I noticed that in current trunk (v1.4, IIUC) things have
> >> changed and libtool symbols are not externally available. Again, I
> >> understand the reason and acknowledge that such change is a really
> >> good thing. However, this change has broken all my hackery for
> >> dlopen()ing libmpi before the call to MPI_Init().
> >>
> >> Is there any chance that libopen-pal could provide some properly
> >> prefixed (let say, using "opal_" as a prefix) wrapper calls to a
> >small
> >> subset of the libtool ltdl API? The following set of wrapper calls
> >> would is the minimum required to properly load libmpi in a portable
> >> manner and cleanup resources (let me abuse of my previous suggestion
> >> and add the opal_ prefix):
> >>
> >> opal_lt_dlinit()
> >> opal_lt_dlexit()
> >>
> >> opal_lt_dladvise_init(a)
> >> opal_lt_dladvise_destroy(a)
> >> opal_lt_dladvise_global(a)
> >> opal_lt_dladvise_ext(a)
> >>
> >> opal_lt_dlopenadvise(n,a)
> >> opal_lt_dlclose(h)
> >>
> >> Any chance this request could be considered? I would really like to
> >> have this before any Open MPI tarball get released without libtool
> >> symbols exposed...

Re: [OMPI devel] libtool issue with crs/self

[Ralf Wildenhues]

Hello,

* Josh Hursey wrote on Wed, Aug 05, 2009 at 04:51:59PM CEST:
> I noticed that the "-fvisibility=hidden" option when passed to
> libltdl will cause it to fail in its configure test for:
>  "checking whether a program can dlopen itself"
> This is because the symbol they are trying to look for with dlsym()
> is not postfixed with:
>   __attribute__ ((visibility("default")))
> If I do that, then the test passes correctly.
> 
> I am not sure if this is a configure bug in Libtool or not.

I've brought this up on the Libtool list:


Thanks,
Ralf

Re: [OMPI devel] trunk borked -- my fault

[Ralf Wildenhues]

Hello,

* Jeff Squyres wrote on Tue, Aug 04, 2009 at 11:38:29PM CEST:
> https://svn.open-mpi.org/trac/ompi/changeset/21759 seems to make us
> play well with AC 2.64.  To be honest, I'm not sure why this change
> works, but it does.

First off, the warnings 2.64 spit out were about real issues (that could
have caused unwanted logic with older Autoconf versions, too, but with
2.64 there are a few more cases with consequences).  The fix in 21759
avoids the warnings but likely doesn't do what you would like it to do.

For example, this code in ompi_setup_f77.m4:

  AC_DEFUN([OMPI_PROG_F77],[ 
  AC_PROG_F77([gfortran g77 f77 xlf frt ifort pgf77 fort77 fl32 af77]) 
  ]) 

  AC_DEFUN([OMPI_SETUP_F77],[ 

  # 
  ompi_fflags_save="$FFLAGS" 
  # Strangeness in AC2.64 forces us to require a macro that calls 
  # PROG_FC instead of calling it directly.  Weird. 
  AC_REQUIRE([OMPI_PROG_F77]) 
  FFLAGS="$ompi_fflags_save" 

will cause the contents of the OMPI_PROG_F77 macro to be expanded
completely before the expansion of the OMPI_SETUP_F77 macro.

That means, the lines
  ompi_fflags_save="$FFLAGS"

  FFLAGS="$ompi_fflags_save" 

will not bracket the AC_PROG_F77 call, as they should.  What you should
do is move these lines to the required macro as well.

The patch below ought to fix up these issues, as well as the scope push
macro for C++.  For clarity, you could also put that in another,
separate macro, and AC_REQUIRE that.

Hope that helps.

Cheers,
Ralf


2009-08-11  Ralf Wildenhues  <ralf.wildenh...@gmx.de>

* config/ompi_setup_f90.m4 (OMPI_PROG_FC, OMPI_SETUP_F90):
Move FCFLAGS save/restore wrap in OMPI_PROG_FC.
* config/ompi_setup_f77.m4 (OMPI_PROG_F77, OMPI_SETUP_F77):
Likewise for FFLAGS.
* config/ompi_setup_cxx.m4 (OMPI_SETUP_CXX)
(_OMPI_SETUP_CXX_COMPILER_HELPER, _OMPI_SETUP_CXX_COMPILER):
Fold OMPI_VAR_SCOPE_PUSH and AC_PROG_CXX{,CPP} calls into a
new helper macro and require that, to fix semantics with respect
to AC_REQUIRE.

Index: config/ompi_setup_f90.m4
===
--- config/ompi_setup_f90.m4(revision 21794)
+++ config/ompi_setup_f90.m4(working copy)
@@ -42,7 +42,9 @@
 # This macro is necessary because PROG_FC is REQUIREd by multiple
 # places in SETUP_F90.
 AC_DEFUN([OMPI_PROG_FC],[
+ompi_fcflags_save="$FCFLAGS"
 AC_PROG_FC([gfortran f95 fort xlf95 ifort ifc efc pgf95 lf95 f90 xlf90 
pgf90 epcf90])
+FCFLAGS="$ompi_fcflags_save"
 ])dnl

 AC_DEFUN([OMPI_SETUP_F90],[
@@ -86,11 +88,9 @@
 # list of 95 and 90 compilers and use it here.
 #

-ompi_fcflags_save="$FCFLAGS"
 # Strangeness in AC2.64 forces us to require a macro that calls
 # PROG_FC instead of calling it directly.  Weird.
 AC_REQUIRE([OMPI_PROG_FC])
-FCFLAGS="$ompi_fcflags_save"
 if test -z "$FC"; then
 AC_MSG_WARN([*** Fortran 90/95 bindings disabled (could not find 
compiler)])
 OMPI_WANT_F90_BINDINGS=0
Index: config/ompi_setup_cxx.m4
===
--- config/ompi_setup_cxx.m4(revision 21794)
+++ config/ompi_setup_cxx.m4(working copy)
@@ -47,11 +47,20 @@
 _OMPI_CXX_CHECK_2D_CONST_CAST
 ])

+# _OMPI_SETUP_CXX_COMPILER_HELPER
+# ---
+AC_DEFUN([_OMPI_SETUP_CXX_COMPILER_HELPER], [
+OMPI_VAR_SCOPE_PUSH(ompi_cxx_compiler_works)
+ompi_cxxflags_save="$CXXFLAGS"
+AC_PROG_CXX
+AC_PROG_CXXCPP
+CXXFLAGS="$ompi_cxxflags_save"
+])
+
 # _OMPI_SETUP_CXX_COMPILER()
 # --
 # Setup the CXX compiler
 AC_DEFUN([_OMPI_SETUP_CXX_COMPILER],[
-OMPI_VAR_SCOPE_PUSH(ompi_cxx_compiler_works)

 # There's a few cases here:
 #
@@ -66,11 +75,8 @@
 # both found a c++ compiler and want the C++ bindings (i.e., either
 # case #1 or #3)

-ompi_cxxflags_save="$CXXFLAGS"
-AC_REQUIRE([AC_PROG_CXX])
-AC_REQUIRE([AC_PROG_CXXCPP])
+AC_REQUIRE([_OMPI_SETUP_CXX_COMPILER_HELPER])
 BASECXX="`basename $CXX`"
-CXXFLAGS="$ompi_cxxflags_save"

 AS_IF([test "x$CXX" = "x"], [CXX=none])
 set dummy $CXX
Index: config/ompi_setup_f77.m4
===
--- config/ompi_setup_f77.m4(revision 21794)
+++ config/ompi_setup_f77.m4(working copy)
@@ -41,7 +41,9 @@
 # This macro is necessary because PROG_FC is REQUIREd by multiple
 # places in SETUP_F90.
 AC_DEFUN([OMPI_PROG_F77],[
+ompi_fflags_save="$FFLAGS"
 AC_PROG_F77([gfortran g77 f77 xlf frt ifort pgf77 fort77 fl32 af77])
+FFLAGS="$ompi_fflags_save"
 ])

 AC_DEFUN([OMPI_SETUP_F77],[
@@ -58,11 +60,9 @@
 # Always run this test, even if fortran isn't wanted so that F77 has
 # value for the Fint tests
 #
-ompi_fflags_save="$FFLAGS"
 # Strangeness in AC2.64 forces us

Re: [OMPI devel] Shared library versioning

[Ralf Wildenhues]

* Jeff Squyres wrote on Sat, Jul 25, 2009 at 07:59:25PM CEST:
> On Jul 25, 2009, at 12:59 PM, Iain Bason wrote:
> >> We have talked many times about doing proper versioning for
> >> OMPI's .so libraries (e.g., libmpi.so -- *not* our component DSOs).
> >
> >Forgive me if this has been hashed out, but won't you run into trouble
> >by not versioning the components?
> 
> This is a good question; it has been discussed a few times, but it's
> good to get it here on the web archives.  More below.

Blissfully unaware of any prior discussion on this topic, I still cannot
help but add my two cents here. :-)

> >What happens when there are
> >multiple versions of libmpi installed?  The user program will pick up
> >the correct one because of versioning, but how will libmpi pick up the
> >correct versions of the components?
> 
> Even with this shared library versioning, you will still only really
> install one OMPI per directory tree anyway.  The library versioning
> won't allow you to install N different versions of OMPI in a single
> tree because of multiple things:
> 
> - support files are not versioned (e.g., show_help text files)
> - include files are not versioned (e.g., mpi.h)
> - OMPI's DSOs actually are versioned, but more work would be needed
> in this area to make that versioning scheme work in real world
> scenarios
> - ...and probably some other things that I'm not thinking of...

You can probably solve most of these issues by just versioning the
directory names where you put the files; and with some luck, some
downstream distribution can achieve this by merely passing a bunch of
--foodir=... options to configure.

That is to say, for modules that you actually dlopen, you could just
ensure that the runtime loader doesn't find these by default, and then
add some versioned directory to its search path for the right version of
these files.

(OMPI probably does all this already; can you tell I have no idea
whether it does?)

Cheers,
Ralf

[OMPI devel] faster autogen.sh

[Ralf Wildenhues]

As part of a shameless advertising move I suggest this patch.
While it won't make autogen.sh exactly fast, for me it shaves
half a minute off its runtime.

Cheers,
Ralf

HACKING: recommend parallel automake 1.11 execution.

Index: HACKING
===
--- HACKING (revision 21262)
+++ HACKING (working copy)
@@ -197,7 +197,15 @@

Running autogen.sh may take several minutes.  It's not very
exciting to watch.  :-)
+   If you have an SMP system, say with 4 processors, be sure to use
+   Automake 1.11 or newer and enable threaded execution before running
+   autogen.sh (you can again put this in your shell startup files):

+  # For bash/sh:
+  export AUTOMAKE_JOBS=4
+  # For csh/tcsh:
+  set AUTOMAKE_JOBS 4
+
5a. You generally need to run autogen.sh whenever the top-level
file "configure.ac" changes, or any files in the config/
directory change (the config/ directory is where a lot of

Re: [OMPI devel] Build failures on trunk? r21235

[Ralf Wildenhues]

Hi Brian,

* Brian W. Barrett wrote on Thu, May 14, 2009 at 08:22:58PM CEST:
>
> Actually, I think that was something else.  Today, libopen-rte.la lists  
> libopen-pal.la as a dependency and libmpi.la lists libopen-rte.la.  I had 
> removed the dependency of libmpi.la on libopen-pal.la because it was  
> causing libopen-pal.so to be listed twice by libtool, which was causing  
> problems.

That's weird, and shouldn't happen (the problems, that is).  Do you have
a pointer for them?

Thanks,
Ralf

Re: [OMPI devel] Build failures on trunk? r21235

[Ralf Wildenhues]

Hello,

* Jeff Squyres wrote on Thu, May 14, 2009 at 07:56:24PM CEST:
> On May 14, 2009, at 1:46 PM, Ralf Wildenhues wrote:
>
>> A more permanent workaround could be in OpenMPI to list each library
>> that is used *directly* by some other library as a dependency.  Sigh.
>
> We actually took pains to *not* do that; we *used* to do that and  
> explicitly took it out.  :-\  IIRC, it had something to do with  
> dlopen'ing libmpi.so...?

Admittedly, I didn't look at Open MPI in detail before writing my
previous reply.  So it would be nice to know the outcome of the
workaround anyway (I do have a Debian here, but different Libtool
versions and little time), there could also be another genuine bug
hiding there.  Dlopening sounds like Debian Libtool issue though,
and one worthy of a Debian bug report (because that is not intended
by them to fail).

Thanks,
Ralf

Re: [OMPI devel] Build failures on trunk? r21235

[Ralf Wildenhues]

Hello,

Ashley, did you rebootstrap with Debian's Libtool?

They enable link_all_deplibs=no in their Libtool which changes some
things and can cause issues like this.  Can't hurt to open a Debian
bug report about it (targeted against libtool) so they know this issue
exists.

Can you try working around it by setting link_all_deplibs to "yes",
then rebuilding all the libraries?  Like this, done in the top build
directory with your current build tree:
  find . -name libtool | xargs \
sed -i 's/^\(link_all_deplibs=\).*//'
  find . -name \*.la | xargs ./libtool --mode=clean rm -f
  make

If that does not work, then I'd be very interested in what the failure
would look at that point.

A more permanent workaround could be in OpenMPI to list each library
that is used *directly* by some other library as a dependency.  Sigh.
Or fix Debian Libtool.

Cheers,
Ralf

* Jeff Squyres wrote on Thu, May 14, 2009 at 07:28:47PM CEST:
> Hmm.  This may not be pilot error.  I build OMPI with a pre-installed  
> OMPI all the time and they don't conflict during the build (i.e., the  
> building OMPI always uses the libopen-rte and libopen-pal from the build 
> tree, not the install tree).  Here's my link lines for ompi_info:
>
> /bin/sh ../../../libtool --tag=CXX   --mode=link g++  -g -Wall -Wundef  
> -Wno-long-long -finline-functions -pthread  -export-dynamic   -o  
> ompi_info components.o ompi_info.o output.o param.o version.o ../../../ 
> ompi/libmpi.la -lnsl  -lutil -lm
> libtool: link: g++ -g -Wall -Wundef -Wno-long-long -finline-functions - 
> pthread -o .libs/ompi_info components.o ompi_info.o output.o param.o  
> version.o -Wl,--export-dynamic  ../../../ompi/.libs/libmpi.so /users/ 
> jsquyres/svn/ompi/orte/.libs/libopen-rte.so /users/jsquyres/svn/ompi/ 
> opal/.libs/libopen-pal.so -ldl -lnsl -lutil -lm -pthread -Wl,-rpath - 
> Wl,/home/jsquyres/bogus/lib
>
> Notice that libopen-rte.os and libopen-pal.so are explicitly mentioned  
> by absolute path name.  Yours weren't.  I wonder why...?
>
>
> On May 14, 2009, at 12:41 PM, Ashley Pittman wrote:
>
>>
>> Libtool is 2.2.6.  I use debian unstable so it's normally fairly
>> up-to-date, I suppose it's not impossible that a debian update has
>> broken things now that I think of it.
>>
>> I normally build in memfs for speed and have just rebooted my machine
>> now, a full rebuild has failed again with the same errors.
>>
>> All three symbols are shown as B according to nm so they should be
>> available.
>>
>> Actually further testing shows it's user error again, if I remove the
>> current install then the build succeeds, it must have been pickings up
>> the libopen-pal from the install location rather than from the current
>> build.
>>
>> Ashley Pittman,

Re: [OMPI devel] Fwd: RFC: proposed GPLv3 license exception draft

[Ralf Wildenhues]

Hello Ralph,

* Ralph Castain wrote on Sat, Apr 25, 2009 at 05:09:01PM CEST:
> Just to be clear, Ralf - I'm not advocating that we change build  
> systems. I agree it has been a good relationship, and your participation 
> has been welcome and extremely helpful.

Understood; and thanks!

> My point was only that the GPL continues to evolve and seems to be  
> growing more aggressive in its "viral" clauses, which makes it harder to 
> work with those packages without getting "assimilated", as the Borg  
> would say.

(The following is of course my sole opinion only.)

Well, of course I cannot tell you how to view or judge the evolution of
the GPL either; and I understand that the anti-tivoization clause in
GPLv3 is one of the developments you seem to be hinting at.  I don't
claim that to be different from how you see it.  Of course, the FSF
would formulate that differently: they saw a "subversion" of the GPLv2
in intent if not in the letter of the license.  So they closed that
loophole.

However, with respect to the exception clauses that have been written
for GCC, and are being written for Autoconf, at least the *intent* has
not been to become any more "viral" than before at all.  For example,
the GCC Exception, as far as I understand it, has opened up the way to
write plugins for GCC; something, which the FSF did not want to allow
to happen at all with the GPLv2 Exception.

With the current Autoconf draft Exception, there are two goals: one, to
merely reformulate the current GPLv2 Exception in terms of GPLv3 Section
"7. Additional Terms", which has formalized the whole exception business
a bit.  Second, to address in some way the apparent loophole that one
could take the whole of Autoconf and clone it by the construction that
I have mentioned before.  Note that this "loophole" is something the
lawyers considered possible from reading the text of the old GPLv2
Exception, but it would pretty clearly already have been against the
intent of that old Exception (i.e., the loophole would be considered
a hack by any means).

To be honest, I don't see the tendency to get "more viral", rather, the
desire to clarify things where intention and letter of licenses differ.
I must admit though, that the newer license texts are longer, more
complicated, and may be intimidating on that basis along, which should
be taken into account, too.

Cheers,
Ralf

Re: [OMPI devel] Fwd: RFC: proposed GPLv3 license exception draft

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Sat, Apr 25, 2009 at 01:27:24PM CEST:
> We OMPI developers ask people to send the stdout/stderr of configure and 
> their config.log to us to help figure out problems 
> (http://www.open-mpi.org/community/help/).  As I understand your 
> explanations, this is still perfectly fine -- these scenarios are long 
> after running AC/AM/LT, so all that data is covered by the exception and 
> is therefore effectively licensed under Open MPI's license.
>
> Is that correct?

Yes, that is correct.

>> 2) While developing a complex autotools-based build system such as the
>> one that Open MPI uses, it might be quite helpful to peek at internals
>> of Autoconf macros, and in some cases copy constructs from them and  
>> use
>> them in Open MPI's macros, to the point that it's unclear whether one
>> code is derived from another in a legally significant way.
>>
>> In such a case, I'm personally not sure what the *desired* stance of
>> the FSF would be about the required license of those copied macros
>> (my personal preference would be to allow this, as long as the intent
>> is clear to not create an Autoconf clone).  However, even if the FSF
>> were to intend to make those honor the GPL, then it should still be
>> possible to distribute the produced configure script without
>> restrictions.

> Yes, this is also a good point.  There are a small number of places in  
> OMPI's build system where we are using internal / unpublished AC/AM  
> mechanisms (e.g., global shell variables that have the output of various 
> tests that are not documented).  We *needed* to use these because we 
> deviated a bit from what AC/AM normally provides (obviously not because 
> we're trying to create an AC/AM clone or anything like that).  Are these 
> problematic from a licensing point of view?  (of course, all the standard 
> technical "this isn't part of the published interface and may change at 
> any time" stuff applies as well)

Well, the question whether they are problematic or not, is one where in
the end, only a lawyer can provide a definite answer for you.  Whether
for the current GPLv2 + the simple Autoconf Exception, or with a future
GPLv3 + Exception.

I brought this up now precisely because I'm not quite sure of this
myself, but I'd like the next Autoconf license to be clear here, and
in a way that this works for you.  Please note that I'm not the person
to decide this, in the end, it is the FSF.

> I don't recall where these places are in OMPI's configure system, so I  
> can't cite them easily, but I'm sure we have some that have crept in  
> over the years.  It might be difficult to find them all and root them  
> out, if they are problematic, license-wise.

I don't actually think there is much relevant code of this form at all
in Open MPI.  I haven't done an analysis, though.  And if there would
turn out to be relevant portions, _and_ the license question can't be
cleared up, then I'd see it as next step on my TODO list to help redo
the Open MPI macros in a clean fashion.

>> Of course I am not in a position to tell you what build system to use,
>> but in my view, both autotools and Open MPI have profited quite a bit
>> from each other (I hope!), in that the former has gained support for
>> several new systems since, squashed lots of bugs, and the latter has
>> been a very good stress test example, and as a result, the former now
>> has several improvements for large packages (faster config.status,
>> less bloat in Makefile.in files, threaded automake execution) from  
>> which the latter may profit.

> Agreed.
>
> At one point, we investigated switching away from AM/LT and using scons 
> as a building tool (still using AC as the configuring tool).  As a 
> technology, there were certain distinct advantages to using scons --  
> some aspects of it were downright cool, actually.

I hardly know scons.  What's cool about it that autotools don't have?

Just in case less verbose build output (a la Linux kernel builds) is one
of them, Automake-1.11 will have that (and its release is only waiting
for the license stuff to be resolved).

> 2. The support we get from Ralf and the AC/AM/LT community has been  
> great; I don't know if we'd get that level of support from the scons  
> community

Thanks!

Cheers,
Ralf

Re: [OMPI devel] Fwd: RFC: proposed GPLv3 license exception draft

[Ralf Wildenhues]

Hello Open MPI developers,

* Paul H. Hargrove wrote on Fri, Apr 24, 2009 at 09:17:20PM CEST:
>  I think your specific example of public posting of the debug or verbose 
> output might be a valid concern, but perhaps not exactly in the way 
> you've stated it.  The act of "distributing" the debug/verbose output is 
> not forbidden, but distributing it under a *non-GPL* license is.  If by 
> posting the debug or verbose output one was now required to offer, under 
> GPL terms, *all* the source code that generated said output, then I'd say 
> we have a problem when configure.ac and acinclude.m4 are from a non-GPL 
> project such as OMPI.

> Ralf Wildenhues,
>  While I believe Jeff is the one that brought this discussion to  
> ompi-devel, I know you've posted on ompi-devel in the past.  Are you  
> seeing this?

Yes, I am casually reading this list.

>  Do you have a response to Ralph's concern, or can you  
> bring this to the attention of those who would?  Perhaps we are  
> rehashing a discussion already settled on some list we don't read?

Well, it brings up an interesting point; part of this has been discussed
already, part hasn't.  For those interested, most of the other
discussion has taken place on the autoconf list as part of this thread:
<http://lists.gnu.org/archive/html/autoconf/2009-04/msg00081.html>

For the points brought up here, allow me to give a bit of background:

The whole language about "verbosity" and "non-debugging" output was
added to the Exception in the first place, in order to prevent something
like this to happen: somebody modifies autoconf to output all of the
macro input files that are part of the Autoconf software package itself;
then somebody else uses that output to create an Autoconf clone under
any license.  It was considered that this might be possible with the
current GPLv2 + Exception.  Note this all about somebody who wants to
create another "Autoconf", not about normal users of it.

Now, as the Exception is formulated currently, it certainly has the
chance to also impact normal users of Autoconf.  Finding out whether
this may be a problem for our users is one of the very reasons we posted
this draft Exception as an RFC; we would like to ensure that our users
are not impacted negatively by it.  This includes that it should
continue to be possible to produce configure scripts for non-free
software and distribute it alongside that, even without source code.

Here's the way I see it (and IANAL):

As long as you only deal with the configure script and output from that,
may that be debugging output, verbose output, whatever, you can do
anything with it that you want.  This Exception only ever deals with
what is done with the output of "autoconf", i.e., whatever happens
when "autogen.sh" is run on the Open MPI tree.

Still, I can see that there may be two problematic issues, and I would
like to thank you for bringing them up; I will talk to the FSF legal
dept. about them.  Here's my summary of them, formulated in a way that I
will present them to the FSF; please correct me if I have misrepresented
or forgotten anything, thanks.

1) While developing your build system, you might have some problem which
is Autoconf-related.  Autoconf developers ask you to provide output
from, say,
  autoconf --verbose

and now, since you are going to publish it on a mailing list such as the
bug-autoconf one, thus effectively distributing it, there are
restrictions on the produced text.  Since the output will likely depend
on both code from Autoconf, and also code from your build system, can
this now require you to provide your build system bits under a
GPL-compatible license?

2) While developing a complex autotools-based build system such as the
one that Open MPI uses, it might be quite helpful to peek at internals
of Autoconf macros, and in some cases copy constructs from them and use
them in Open MPI's macros, to the point that it's unclear whether one
code is derived from another in a legally significant way.

In such a case, I'm personally not sure what the *desired* stance of
the FSF would be about the required license of those copied macros
(my personal preference would be to allow this, as long as the intent
is clear to not create an Autoconf clone).  However, even if the FSF
were to intend to make those honor the GPL, then it should still be
possible to distribute the produced configure script without
restrictions.

> Ralph Castain wrote:
>> Frankly, this all seems absurd to me. The GPL continues to grow in its  
>> unfriendliness. Perhaps it is time we reconsider our use of these  
>> tools - we had given some consideration in the past to moving, and  
>> maybe we need to do so again.

Of course I am not in a position to tell you what build system to use,
but in my view, both autotools and Open MPI have profited quite a bit
from each other (I hope

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r20568

[Ralf Wildenhues]

Hello,

* Jeff Squyres wrote on Tue, Feb 17, 2009 at 07:01:01PM CET:
> On Feb 17, 2009, at 11:18 AM, George Bosilca wrote:
>
>> I guess that if the free function supports the NULL pointer we should 
>> do the same...
>
> I'll agree with that if we know for sure that free(NULL) is universally 
> supported.  You mentioned "a few man pages" -- how universal is this 
> support?

The Cray T3E used to barf on free(NULL).  I haven't personally
encountered anything since.  C89 requires it to work, and GNU
software generally assumes it to work since a couple of years.

Cheers,
Ralf

Re: [OMPI devel] [Pkg-openmpi-maintainers] Building with rpath disabled

[Ralf Wildenhues]

Hello Jeff,

* Jeff Squyres wrote on Tue, Jan 13, 2009 at 03:39:28PM CET:
> On Jan 13, 2009, at 4:54 AM, Manuel Prinz wrote:
>>
>> You have to pass --disable-rpath explicitely. Building with rpath is
>> still the default. I verified by building without passing any option  
>> to configure and the resulting libs contained an rpath.
> Just for the web archives: per some off-list discussion, we decided not 
> to take the patch because the Debian guys have a simpler workaround for 
> what they want.

Just for the web archives: how was that done?  I mean, some other distro
just could have the same issue, and you (or they) wouldn't have to
reinvent the same solution again.

Thanks,
Ralf

Re: [OMPI devel] make dependency problem?

[Ralf Wildenhues]

Hello Eugene, Jeff,

the interesting data points here would be the dependency tracking scheme
selected by automake (CCDEPMODE in ompi/mca/bml/r2/Makefile), with and
without --enable-dependency-tracking passed to configure.

Also, the contents of ompi/mca/bml/r2/.deps/bml_r2.Plo, after ensuring
that a dependency tracking scheme other than "none" is selected, and
then removing bml_r2.o once and rebuilding.

Also, which compiler, and version, and which Solaris release?

Thanks,
Ralf

* Jeff Squyres wrote on Tue, Dec 02, 2008 at 01:58:49PM CET:
> Weird -- the exact opposite happens for me (if I touch btl.h, it  
> automatically rebuilds oodles of stuff, to include bml_r2.c).
>
> I have dim recollections of Automake disabling dependency tracking by  
> default on Solaris+Solaris compilers (I don't remember the exact issue  
> -- perhaps it was before AM supported the dependency format of the  
> Solaris compilers...?).  Have you tried configuring with --enable- 
> dependency-tracking?
>
>
>
> On Nov 29, 2008, at 6:45 PM, Eugene Loh wrote:
>
>> I was playing with OMPI and I noticed that if I modified btl.h,  
>> bml_r2.c did not automatically get rebuilt, even though it includes  
>> btl.h.  This caused me all sorts of unnecessary debugging troubles.   
>> In the end, just touching bml_r2.c was enough... it caused bml_r2.c to 
>> be recompiled and to see the changes in btl.h.
>>
>> So, question:
>>
>> Given that bml_rc.2 includes btl.h, wouldn't the proper make  
>> dependencies cause bml_rc.c to be recompiled whenever btl.h is  
>> touched?
>>
>> Again, it appears that this is not happening and that strikes me as a 
>> problem -- for someone out there to fix.  :^)

Re: [OMPI devel] Forwarding SIGTSTP and SIGCONT

[Ralf Wildenhues]

Hello Rolf,

* Rolf Vandevaart wrote on Fri, Dec 05, 2008 at 08:00:42PM CET:
>
> One problem is that with SIGTSTP no longer delivering a stop signal to  
> mpirun, one cannot CTRL-Z at their terminal to stop mpirun.  I am trying  
> to figure out how big a problem that is.

Why not first deal with sending STOP to the other processes, and when
done with that, sending a STOP to mpirun, so it stops as well?

BTW, sending STOP to processes rather than TSTP will prevent them from
being able to do similar to user processes of their own.  Dunno how
useful or practical that is for OpenMPI users.

Cheers,
Ralf

Re: [OMPI devel] RFC: Add SunStudio/Libtool helper script for post-configure

[Ralf Wildenhues]

Hello Ralph,

* Ralph Castain wrote on Mon, Nov 24, 2008 at 02:39:13PM CET:
> On Nov 23, 2008, at 1:19 AM, Ralf Wildenhues wrote:
>>
>> While I suppose your patch works, I think in similar situations,  
>> OpenMPI has resorted to patching input files to configure (like
>> aclocal.m4 or ltmain.sh).  Search autogen.sh for instances of
>> 'patch'.
>
> Interesting observation - I wonder if that would explain why I recently 
> started seeing an "aclocal.m4.rej" file every time I re-gen the trunk?
>
> Maybe one of those patches has an error in it?

More likely, one or more of the patches are not needed with the Libtool
version you're using, because they have already been integrated into
that version.

Ergo, the advantage of patching input files here is that there is a
clean upgrade path.  Except for *.rej files (which autogen.sh could
clean up after itself, I guess).

Cheers,
Ralf

Re: [OMPI devel] RFC: Add SunStudio/Libtool helper script for post-configure

[Ralf Wildenhues]

* Ethan Mallove wrote on Fri, Nov 21, 2008 at 09:01:56PM CET:
> On Fri, Nov/21/2008 01:02:12PM, Ralf Wildenhues wrote:
> > IMHO OpenMPI can use
> > the solaris_use_stlport4=yes until such a functionality is in place.
> 
> Nice. This workaround works. I don't suppose there's a similar
> workaround to unset "wl" in the FC libtool section? If not, I think we
> still need the heinous post-configure workaround script. Otherwise,
> since there won't be a stable Libtool that contains the Sun Fortran
> fix for a while, I propose the attached patch.

While I suppose your patch works, I think in similar situations, OpenMPI
has resorted to patching input files to configure (like aclocal.m4 or
ltmain.sh).  Search autogen.sh for instances of 'patch'.

(This is merely an observation; I am not an OpenMPI developer.)

Cheers,
Ralf

Re: [OMPI devel] RFC: Add SunStudio/Libtool helper script for post-configure

[Ralf Wildenhues]

Hello Ethan, all,

* Ethan Mallove wrote on Thu, Nov 20, 2008 at 10:33:08PM CET:
> On Thu, Nov/20/2008 07:00:31PM, Ralf Wildenhues wrote:
> > 
> > Ah, ok.  Please try the patch below instead of yours, thanks.
> 
> Your patch seems to work, though I get this:
> 
>libtool: Version mismatch error.  This is libtool 2.2.7a, but the
>libtool: definition of this LT_INIT comes from libtool 2.2.6.
>libtool: You should recreate aclocal.m4 with macros from libtool 2.2.7a
>libtool: and run autoconf again.
> 
> I take it the above error will occur if I have two different libtools
> in my PATH?

No.  That means the macro files that were picked up were from Libtool
2.2.6, while the ltmain.sh file is from 2.2.7a.

> This comment could be a little misleading because the same is true for
> Sun Fortran 8.1 and 8.2:
> 
>   # Sun Fortran 8.3 passes all unrecognized flags to the linker

OK.  I think we simply didn't have any other version to test at the time
this was written.  We usually list the version somewhere so we can check
for version-specific issues, should they later show up.

I will update the comment to list '8.1 through 8.3', when I commit your
patch (sometime this weekend); thanks for testing it.

> I don't know of a version of Sun Fortran that accepts -Wl the way GNU
> Fortran does. I will let you know if I find one.

Thanks.

> > > I'm still running into the Cstd/stlport4 issue with 2.2.6. That is,
> > > this line appears in the libtool script:
> > > 
> > >   postdeps="-library=Cstd -library=Crun"
> >
> > Do you have the string " -library=stlport4 " in $CXX $CXXFLAGS?
> > If not, then how can Libtool detect that you use stlport?
> 
> Ok. When I use -library=stlport4, I get libstlport linked to
> libmpi_cxx, instead of libCstd. Doesn't that then lock the user into
> having to use stlport4 when we want them to be able to use either Cstd
> or stlport4?

Hmm, yes, it does.  It is a bit of a problem to let libtool avoid either
standard C++ library in general: with shared libraries or even
dlopen'able modules, this can result in undefined symbols at run time.

As the code is currently written in libtool.m4, there is an undocumented
way which you can use to get the effects of adding neither library: set
solaris_use_stlport4=yes.  You can use this, either as argument to
configure, or set it inside configure.ac (or a macro) so that it is
expanded before AC_PROG_LIBTOOL.

However, as it is undocumented, there is no guarantee that it will
continue to work indefinitely.  What Libtool should instead do future is
provide some configure flag to allow to specify that no C++ standard
library is to be linked in by default.  That would help for a couple of
different setups with other compilers as well.  IMHO OpenMPI can use
the solaris_use_stlport4=yes until such a functionality is in place.

Cheers, and thanks,
Ralf

Re: [OMPI devel] RFC: Add SunStudio/Libtool helper script for post-configure

[Ralf Wildenhues]

Our previous mails overlapped, sorry about that.

* Ethan Mallove wrote on Thu, Nov 20, 2008 at 06:52:09PM CET:
> 
> The above appears to be looking for a Fortran version string from the
> C compiler, but it wouldn't match our version string anyway:
> 
>   $ f90 -V
>   f90: Sun Ceres Fortran 95 8.3 SunOS_sparc 2008/01/28

Ah, ok.  Please try the patch below instead of yours, thanks.
Do you mind being added to libtool/THANKS (includes email address)?

> I'm still running into the Cstd/stlport4 issue with 2.2.6. That is,
> this line appears in the libtool script:
> 
>   postdeps="-library=Cstd -library=Crun"

Do you have the string " -library=stlport4 " in $CXX $CXXFLAGS?
If not, then how can Libtool detect that you use stlport?

Thanks,
Ralf

* libltdl/m4/libtool.m4 (_LT_COMPILER_PIC) [ linux ]: Also
match `Sun Ceres Fortran' compiler; reorder with C compiler
matching.
* THANKS: Update.
Report by Ethan Mallove.

diff --git a/libltdl/m4/libtool.m4 b/libltdl/m4/libtool.m4
index 7fbf965..d90c4f4 100644
--- a/libltdl/m4/libtool.m4
+++ b/libltdl/m4/libtool.m4
@@ -3947,17 +3947,17 @@ m4_if([$1], [CXX], [
;;
   *)
case `$CC -V 2>&1 | sed 5q` in
-   *Sun\ C*)
- # Sun C 5.9
+   *Sun\ F* | *Sun*Fortran*)
+ # Sun Fortran 8.3 passes all unrecognized flags to the linker
  _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC'
  _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic'
- _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,'
+ _LT_TAGVAR(lt_prog_compiler_wl, $1)=''
  ;;
-   *Sun\ F*)
- # Sun Fortran 8.3 passes all unrecognized flags to the linker
+   *Sun\ C*)
+ # Sun C 5.9
  _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC'
  _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic'
- _LT_TAGVAR(lt_prog_compiler_wl, $1)=''
+ _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,'
  ;;
esac
;;

Re: [OMPI devel] RFC: Add SunStudio/Libtool helper script for post-configure

[Ralf Wildenhues]

Hello Ethan,

* Ethan Mallove wrote on Wed, Nov 19, 2008 at 09:24:16PM CET:
> 
> I'm seeing the same issue with the faulty "wl" Libtool
> variable in 2.2.6 with Linux SunStudio:

That's really weird, because this change should have fixed that:


>   make[5]: Entering directory `ompi/mpi/f90'
>   /bin/sh ../../../libtool   --mode=link f90 -I../../../ompi/include 
> -I../../../ompi/include -M. -I. -I../../../ompi/mpi/f90  -m32 -xO5  
> -export-dynamic-o libmpi_f90.la -rpath /opt/SUNWhpc/HPC8.1/sun/lib mpi.lo 
> mpi_sizeof.lo mpi_comm_spawn_multiple_f90.lo mpi_testall_f90.lo 
> mpi_testsome_f90.lo mpi_waitall_f90.lo mpi_waitsome_f90.lo mpi_wtick_f90.lo 
> mpi_wtime_f90.lo   ../../../ompi/libmpi.la -lnsl -lutil  -lm
>   libtool: link: f90 -G  .libs/mpi.o .libs/mpi_sizeof.o 
> .libs/mpi_comm_spawn_multiple_f90.o .libs/mpi_testall_f90.o 
> .libs/mpi_testsome_f90.o .libs/mpi_waitall_f90.o .libs/mpi_waitsome_f90.o 
> .libs/mpi_wtick_f90.o .libs/mpi_wtime_f90.o   -Wl,-rpath -Wl,ompi/.libs 
> -Wl,-rpath 
> -Wl,/tmp/mtt-scratch-patch-libtool-for-sun-studio/mpi-install/HFfN/src/ompi-ct8.1-v1.3-sandbox/orte/.libs
>  -Wl,-rpath 
> -Wl,/tmp/mtt-scratch-patch-libtool-for-sun-studio/mpi-install/HFfN/src/ompi-ct8.1-v1.3-sandbox/opal/.libs
>  -Wl,-rpath -Wl,/opt/SUNWhpc/HPC8.1/sun/lib 
> -L/tmp/mtt-scratch-patch-libtool-for-sun-studio/mpi-install/HFfN/src/ompi-ct8.1-v1.3-sandbox/orte/.libs
>  
> -L/tmp/mtt-scratch-patch-libtool-for-sun-studio/mpi-install/HFfN/src/ompi-ct8.1-v1.3-sandbox/opal/.libs
>  ../../../ompi/.libs/libmpi.so 
> /tmp/mtt-scratch-patch-libtool-for-sun-studio/mpi-install/HFfN/src/ompi-ct8.1-v1.3-sandbox/orte/.libs/libopen-rte.so
>  opal/.libs/libopen-pal.so -ldl -lnsl -lutil -lm  -m32   -mt -Wl,-soname 
> -Wl,libmpi_f90.so.0 -o .libs/libmpi_f90.so.0.0.0
>   f90: Warning: Option -Wl,-rpath passed to ld, if ld is invoked, ignored 
> otherwise
[...]

Please post the output of
  f90 -V

and the output of
  ./libtool --tag=FC --config

to see what's going on.

Thanks,
Ralf

Re: [OMPI devel] RFC: merge windows branch into trunk

[Ralf Wildenhues]

I'm probably going to come across as ignorant, and I'm
obviously biased on this matter, but:

* Tim Mattox wrote on Thu, Nov 20, 2008 at 02:53:13PM CET:
> 
> Although I don't use windows myself, I appreciate your
> and others' efforts to expand the number of platforms
> we can run on.  Great work!

what keeps you from using the autotools-based build system
with MSVC?  All you should need is a wrapper like cccl.

Cheers,
Ralf

Re: [OMPI devel] RFC: Add SunStudio/Libtool helper script for post-configure

[Ralf Wildenhues]

Hello Ethan,

* Ethan Mallove wrote on Wed, Nov 19, 2008 at 04:11:23PM CET:
> There are a couple issues with SunStudio and Libtool:

Which Libtool version are you using?  If not 2.2.2 or newer, then please
retry with 2.2.6.  If the problem persists, then we should fix Libtool
rather than patching OpenMPI.  That way, other packages benefit from the
fix as well.

Thanks,
Ralf

Re: [OMPI devel] RFC: libopen-rte --> libompi-rte

[Ralf Wildenhues]

* Jeff Squyres wrote on Thu, Nov 06, 2008 at 08:48:44PM CET:
> On Nov 6, 2008, at 2:42 PM, Ralf Wildenhues wrote:
>
>> Hmm, OpenMPI seems not to use versioning for its shared libraries.
>
> That was on the to-do list for this release, but it just didn't happen.  
> :-\

Oh well.  When you start doing it, please coordinate with distributors;
it's not likely, but some of them may have actually packaged OpenMPI
with versioning, including (one then hopes!) adjusting versions as
needed between releases.  You might want to start out with numbers
upward of that then.

(I've seen Debian do this before.  AFAIK they haven't for OpenMPI,
though.)

>> Do you just declare each release API+ABI-incompatible to each other  
>> release?
>
> MPI API is compatible across releases; it hasn't changed.  We have never 
> [yet] claimed ABI compatible across releases -- we may still do this 
> someday.

FWIW, this task looks quite a bit easier to me than, say, defining an
ABI valid for several MPI implementations.

Cheers,
Ralf

Re: [OMPI devel] RFC: libopen-rte --> libompi-rte

[Ralf Wildenhues]

Hello,

* Jeff Squyres wrote on Thu, Nov 06, 2008 at 07:49:23PM CET:
> On Nov 6, 2008, at 1:13 PM, Brian W. Barrett wrote:
>
>>> WHY: ORTE is really quite specific to OMPI.  We decided long ago  
>>> that ORTE would not split off from OMPI, and it has been  
>>> specifically tailored for OMPI.  Indeed, Ralph has recently been  
>>> expanding "ORTE" to "OMPI Run Time Environment".  So let's name the  
>>> library what it really is.
>>
>> Does it really hurt anything to leave the name as is?

> I understand that some people can't use wrappers, but we added the -- 
> showme options just for these kinds of scenarios.  I had thought that  
> that was sufficient -- are you saying that it's not?

Changing the library name is an incompatible change for all programs and
libraries that are linked against it; i.e., it requires relinking.

Hmm, OpenMPI seems not to use versioning for its shared libraries.
Do you just declare each release API+ABI-incompatible to each other
release?

Cheers,
Ralf

Re: [OMPI devel] adding new functions to a BTL

[Ralf Wildenhues]

Hello Jeff, Eugene,

> Jeff Squyres wrote:
>
>> We use lt_dlopen() to open the plugins (Libtool's wrapper for a   
>> portable dlopen).  It opens all plugins (DSOs) in a private scope.
>> That private scope is kept deep in the OPAL MCA base and not exposed   
>> elsewhere in the code base.  So if you manually dlopen a plugin again,  
>> I'll bet that the linker realizes that that DSO has already been  
>> loaded into the process space and doesn't actually load it again (but  
>> doesn't fail).  So the dlsyms fail because you don't have access to  
>> the private scope from where Libtool originally opened the DSO.

Shouldn't it work to re-dlopen the lib with RTLD_GLOBAL?

Also, recent libltdl should allow you to choose which scope you want in
the first place, local or global, through lt_dladvise.

Hope that helps.

Cheers,
Ralf

Re: [OMPI devel] Should visibility and memchecker abort configure?

[Ralf Wildenhues]

Hello,

if you allow me my 2 cents:

At configure time, it is possible to distinguish between several
different user inputs:
- the user typed --enable-foo,
- the user typed --disable-foo or --enable-foo=no,
- the user typed --enable-foo=ARG (ARG is available for further
  inspection),
- the user used none of the above.

IIUC you have already sorted out the visibility issue by using the last,
and the memchecker issue is about having an incompatible version
installed.  One typical semantics would be to not try to use the feature
at all if --disable-foo was explicitly passed.

Hope that helps.

Cheers,
Ralf

Re: [OMPI devel] gdb libmpi.dylib on Leopard

[Ralf Wildenhues]

* Aurélien Bouteiller wrote on Fri, Sep 19, 2008 at 08:02:40PM CEST:
> Thanks Ralf for the support. I upgraded to libtool 2.2.6 and it didn't  
> solved the problem though. Still looking for somebody to confirm that  
> its working or not working on their Mac.

Did you rerun autogen.sh?  All I know is that your report looks really
similar to  and that
one is apparently solved with Libtool 2.2.6.

If yours is still broken, then some more details would be nice.

Cheers,
Ralf

Re: [OMPI devel] Upgrade GNU auto tools?

[Ralf Wildenhues]

Hello OpenMPI developers,

on Darwin/OS X, recent Libtool has some debugging symbols issue fixed.
I think that was even raised on this list a short while ago.  Also, an
issue with recent Intel compilers on Linux has been fixed.

Absoft support is not yet in Libtool, but Lahey is.

Hope that helps.

Cheers,
Ralf

* Jeff Squyres wrote on Fri, Sep 19, 2008 at 04:07:02PM CEST:
> Yep; there may be no real specific benefit, but it's good to get the  
> most recent versions for a release branch before we freeze it.
>
> It *may* have fixed some Absoft compiler issues; I'm checking with  
> Absoft on this (I didn't see anything specific about it in the release  
> notes, and the data is not entirely clear because a few things changed  
> in Absoft's MTT setup right around the same time).
>
> On Sep 19, 2008, at 10:04 AM, George Bosilca wrote:
>
>> I've been using these versions for some time, basically from the date 
>> they get released. So far, no issues have been raised. However, I do 
>> not see any benefit with these new versions (on Linux and Mac OS X).
>>
>>  george.
>>
>> On Sep 19, 2008, at 9:56 AM, Tim Mattox wrote:
>>
>>> Just an FYI,
>>> Last night I switched the nightly tarball creation for the
>>> trunk and v1.3 to use the latest autoconf and libtool
>>> releases.  AFAIK, everything worked fine.
>>> So, these are the versions of the various gnu tools
>>> we are currently using:
>>> m4-1.4.11
>>> autoconf-2.63
>>> automake-1.10.1
>>> libtool-2.2.6a
>>>
>>> You may want to upgrade your local installs of these for
>>> your own development work so we are all on the same version.
>>>
>>> On Thu, Sep 18, 2008 at 8:24 AM, Jeff Squyres   
>>> wrote:
 Autoconf and Libtool released updates recently (to v2.63 and  
 v2.2.6a,
 respectively).  I have tested these versions with a local trunk  
 checkout and
 they seem to work fine.

 Any objections to moving the nightly trunk and v1.3 tarballs to  
 these
 versions?

 m4=m4-1.4.11
 ac=autoconf-2.63
 am=automake-1.10.1
 lt=libtool-2.2.6a

 (we're still pre-1.3 lockdown, so I figured it was ok to move)

Re: [OMPI devel] gdb libmpi.dylib on Leopard

[Ralf Wildenhues]

Hello Aurélien,

* Aurélien Bouteiller wrote on Wed, Sep 17, 2008 at 06:32:11PM CEST:
> I have been facing a weird problem for several month now (I guess since I 
> upgraded from Tiger to Leopard). I am unable to debug Open MPI using gdb 
> on my mac. The problem comes from gdb not being able to load symbols from 
> the dynamic libraries of Open MPI. I receive a message "warning: Could 
> not find object file "/Users/bouteill/ompi/debug.build/ 
> opal/.libs/libopen-pal.lax/libmca_memory.a/memory_base_close.o" - no  
> debug information available for "../../../../trunk/opal/mca/memory/ 
> base/memory_base_close.c".". As you can see, the path to the object file 
> containing the symbols is not correct. It links to the temporary files 
> expanded during the final stage link. As those files do not exist 
> anymore, gdb gets confused.

I have a vague memory that this is fixed in Libtool 2.2.6.  If you're
using an older version, please retry bootstrapping OpenMPI with that
one.

Cheers,
Ralf

Re: [OMPI devel] MPI ABI on Linux

[Ralf Wildenhues]

* Jeff Squyres wrote on Tue, Sep 09, 2008 at 03:07:24PM CEST:
>> On Sep 9, 2008, at 6:23 AM, Jeff Squyres wrote:
>>> On Sep 9, 2008, at 2:45 AM, Ralf Wildenhues wrote:
>>>
>>>> An MPI ABI will have to be versioned in
>>>> the same way that the API is versioned.  You can have an ABI version
>>>> for each API version though, I guess.
>>>
>>> That is correct.  My first statement wasn't entirely correct --  
>>> "unrelated" is probably not quite the correct word.  Each ABI  
>>> version will be tied to a specific API version.  What I was trying  
>>> to say is that an implementation can be claim to be API compliant,  
>>> even if it's not ABI compliant.

Agreed.

>>>> And of course the MPI C++ ABI will require specifying a C++ ABI
>>>> (which, for Windows, means specifying the compiler and possibly its
>>>> major release number used), but this is venturing off into details.
>>>
>>> Not just Windows, right?

No, but at least on some other systems there is no confusion about which
C++ ABI to pick.

>>> Ditto for Fortran.

Of course.  The devil is in the details.

> I did promise the ABI working group that I would ask the OMPI community 
> to see if anyone wanted to work with Intel on the proof of concept.  
> Let's put a finite end date on the CFP so that I can report back to them: 
> COB this Thursday, Oct 11, 2008.

I'm sure you must mean September not October there.

Are things like timeouts, latencies, and small-message sizes intended
as part of the ABI as well?  IOW, is it expected to be possible to run
one process compiled with OpenMPI and one process with MPICH, and have
them communicate with each other?

Cheers,
Ralf, ready be told that I have no idea what I'm talking about ;-)

Re: [OMPI devel] MPI ABI on Linux

[Ralf Wildenhues]

Hello Jeff, all,

if you allow me my two cents,

* Jeff Squyres wrote on Mon, Sep 08, 2008 at 08:44:28PM CEST:
>
> At the MPI Forum meeting in Dublin, the MPI ABI meeting was... er...  
> shall we say, "spirited."  :-)  Both the benefits and drawbacks of an  
> MPI ABI are widely contended (it's a surprisingly complex topic).

it sounds quite daunting.

> - If it is ever completed, MPI ABI compliance will be a separate entity 
> from the MPI 2.x and 3.x standards.  ABI compliance will be a checkmark 
> for an MPI implementation, but will be unrelated to an implementation's 
> 2.1, 2.2, 3.0, ...etc. compliance.

How can that be possible?   An MPI ABI will have to be versioned in
the same way that the API is versioned.  You can have an ABI version
for each API version though, I guess.

And of course the MPI C++ ABI will require specifying a C++ ABI
(which, for Windows, means specifying the compiler and possibly its
major release number used), but this is venturing off into details.

Cheers,
Ralf

Re: [OMPI devel] TRUNK error ( MAN page ) since r19120

[Ralf Wildenhues]

* Tim Mattox wrote on Mon, Aug 04, 2008 at 03:11:20PM CEST:
> As this thread is making clear, not everyone saw and/or heeded the
> note that we were upgrading the GNU autotools used for building Open MPI's
> nightly tarballs.  See http://www.open-mpi.org/svn/building.php the the 
> version
> matrix.  For your reference, the trunk & 1.3 nightlies are being built
> with these versions, and I recommend everyone upgrade to these:

This has not much to do with autotools versions, but everything
with having a build tree separate from the source tree or not.

The proper fix is to change several references of $(top_srcdir) to
$(top_builddir), to insert into each rule in
ompi_trunk/Makefile.man-page-rules a command like
@$(mkdir_p) `dirname $@`

to have the corresponding directory created in advance.  Alternatively,
you can make files depend on a directory stamp file whose rule creates
the corresponding directory (depending on the directory itself is not
portable).

Also, I see in ompi/tools/*/Makefile.am several instances of
cd SOME_DIR ; make FOO

in makefile rules.  Please fix them to be
cd SOME_DIR && $(MAKE) $(AM_MAKEFLAGS) FOO

so that, if SOME_DIR does not exists, no rogue action is performed,
also that parallel make works as advertised, and that make flags can be
overridden conveniently.

Cheers,
Ralf (who can fix this but then you will have to wait)

[OMPI devel] some configury nits

[Ralf Wildenhues]

Hello,

I tried to build OpenMPI trunk on my Debian stable/testing mix,
and to make things more interesting, today my /bin/sh is dash,
and while gcc is version 4.3.1, g++ remained at 4.1.3.

I needed the following patch in order to get configure to finish
("test ... == ..." is a bash extension).

Then, the build failed with:

| make[6]: Entering directory 
`/tmp/OpenMPI/build/ompi/contrib/vt/vt/tools/vtfilter'
| g++ -DHAVE_CONFIG_H -I. 
-I../../../../../../../ompi-trunk/ompi/contrib/vt/vt/tools/vtfilter -I../.. 
-I../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/otflib 
-I../../extlib/otf/otflib 
-I../../../../../../../ompi-trunk/ompi/contrib/vt/vt/vtlib/ -I../../vtlib   
-fopenmp -DVT_OMP -O3 -DNDEBUG -finline-functions -pthread -MT 
vtfilter-vt_filter.o -MD -MP -MF .deps/vtfilter-vt_filter.Tpo -c -o 
vtfilter-vt_filter.o `test -f 'vt_filter.cc' || echo 
'../../../../../../../ompi-trunk/ompi/contrib/vt/vt/tools/vtfilter/'`vt_filter.cc
| g++ -DHAVE_CONFIG_H -I. 
-I../../../../../../../ompi-trunk/ompi/contrib/vt/vt/tools/vtfilter -I../.. 
-I../../../../../../../ompi-trunk/ompi/contrib/vt/vt/extlib/otf/otflib 
-I../../extlib/otf/otflib 
-I../../../../../../../ompi-trunk/ompi/contrib/vt/vt/vtlib/ -I../../vtlib   
-fopenmp -DVT_OMP -O3 -DNDEBUG -finline-functions -pthread -MT 
vtfilter-vt_filthandler.o -MD -MP -MF .deps/vtfilter-vt_filthandler.Tpo -c -o 
vtfilter-vt_filthandler.o `test -f 'vt_filthandler.cc' || echo 
'../../../../../../../ompi-trunk/ompi/contrib/vt/vt/tools/vtfilter/'`vt_filthandler.cc
| cc1plus: error: unrecognized command line option "-fopenmp"
| cc1plus: error: unrecognized command line option "-fopenmp"
| make[6]: *** [vtfilter-vt_filter.o] Fehler 1

This is because GCC 4.1.3 doesn't know about -fopenmp.  The AX_OPENMP is
called in ACVT_OMP (in ompi/contrib/vt/vt/acinclude.m4) only for the C
compiler, not the C++ one.  Note that the AX_OPENMP is suited to be
called for multiple languages (e.g., wrapped in AC_LANG_PUSH(...) ...
AC_LANG_POP(...)).  However, OpenMPI may decide to not allow for
different compiler settings here, or at least require that all compilers
used do OpenMP.  As I don't know what's desirable, no proposed patch
here.

Cheers,
Ralf

Fix unportable test statements in configure fragments.

Index: opal/mca/memory/ptmalloc2/configure.m4
===
--- opal/mca/memory/ptmalloc2/configure.m4  (Revision 19044)
+++ opal/mca/memory/ptmalloc2/configure.m4  (Arbeitskopie)
@@ -37,7 +37,7 @@
 AM_CONDITIONAL([OMPI_WANT_EXTERNAL_PTMALLOC2],
[test "$enable_ptmalloc2_internal" != "yes"])
 AC_MSG_CHECKING([if ptmalloc2 should be part of libopen-pal])
-AS_IF([test "$enable_ptmalloc2_internal" == "yes"],
+AS_IF([test "$enable_ptmalloc2_internal" = "yes"],
   [AC_MSG_RESULT([yes])], [AC_MSG_RESULT([no])])


Index: ompi/contrib/vt/vt/extlib/otf/acinclude.m4
===
--- ompi/contrib/vt/vt/extlib/otf/acinclude.m4  (Revision 19044)
+++ ompi/contrib/vt/vt/extlib/otf/acinclude.m4  (Arbeitskopie)
@@ -306,7 +306,7 @@
py_version=`$PYTHON -c "from distutils.sysconfig import 
*; \
from string import join; \
print join(get_config_vars('VERSION'))" 2> 
/dev/null`
-   if test "$py_version" == "[None]" -o -z "$py_version"; 
then
+   if test "$py_version" = "[None]" -o -z "$py_version"; 
then
if test -n "$PYTHON_VERSION"; then
py_version=$PYTHON_VERSION
else
@@ -320,7 +320,7 @@
print '-L' + get_python_lib(0,1), \
'-lpython';" 2> /dev/null`$py_version
fi
-   if test ! "$PYTHON_LDFLAGS" == "$py_version"; then
+   if test ! "$PYTHON_LDFLAGS" = "$py_version"; then
AC_MSG_RESULT([$PYTHON_LDFLAGS])
else
AC_MSG_RESULT([no])
Index: ompi/contrib/vt/vt/acinclude.m4
===
--- ompi/contrib/vt/vt/acinclude.m4 (Revision 19044)
+++ ompi/contrib/vt/vt/acinclude.m4 (Arbeitskopie)
@@ -312,7 +312,7 @@
while :
do
$2=`eval echo $var`
-   AS_IF([test $$2 == $var], [break], [var=$$2])
+   AS_IF([test $$2 = $var], [break], [var=$$2])
done
 ])

Re: [OMPI devel] Need v1.3 RM ruling (was: Help on building openmpi with "-Wl, --as-needed -Wl, --no-undefined")

[Ralf Wildenhues]

* George Bosilca wrote on Thu, Jul 24, 2008 at 02:44:04PM CEST:
> I was hoping we can get some hints/help from Ralf, but unfortunately his 
> email address is not CC to the emails and I don't know if he's reading 
> the devel mailing list. I'll resend my email with.

I read the list, but reading latency is anywhere from 5 mins to a couple
of days.

Cheers,
Ralf

Re: [OMPI devel] Need v1.3 RM ruling (was: Help on building openmpi with "-Wl, --as-needed -Wl, --no-undefined")

[Ralf Wildenhues]

* Brian Barrett wrote on Thu, Jul 24, 2008 at 02:28:45PM CEST:
> When I looked at the same problem in LAM, I couldn't get the  
> dependencies right between libraries when only one makefile was used. It 
> worked up until I would do a parallel build, then would die because the 
> libraries weren't ready at the right time.  There's probably a way, but I 
> ended up with Jeff's approach.

I have no idea what Jeff's approach is, and I would not recommend
entering some makefiles more than once, but what you can do is list
some files outside their directory.  I.e, you could have

-- mpi/f77/Makefile_base.am --
libmpi_f77_base_la_SOURCES = mpi/f77/file1.f ...

-- Makefile.am --
include mpi/f77/Makefile_base.am
...

-- mpi/f77/Makefile.am --
# This will appear later in SUBDIRS
libmpi_f77_la_SOURCES = file3.f ...
...

to avoid actually moving around files.  If you find a bug with parallel
make that way, then please report it to the Automake list (but we
haven't seen any Automake bugs in this area for quite a while).

I haven't checked how the profiling stuff fits in this picture.

Just remember that the ordered graph connecting Makefiles (as opposed to
included makefile snippets, or source files) through SUBDIRS entries
should be a tree, and this tree is always traversed bottom up unless '.'
is listed somewhere explicitly.

Cheers,
Ralf

Re: [OMPI devel] Help on building openmpi with "-Wl, --as-needed -Wl, --no-undefined"

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Wed, Jul 23, 2008 at 03:54:50PM CEST:
>
> Is the attached patch what you're talking about?
>
> If so, I'll commit to trunk, v1.2, and v1.3.

Can you verify that it work with a pristine build?  The dependencies as
such look sane to me, also the cruft removal, but I fail to see how
your directory ordering can work:

ompi/mpi/f77/libmpi_f77.la  depends on
ompi/libmpi.la  depends on 
ompi/mpi/f77/libmpi_f77_base.la

The SUBDIRS in ompi/Makefile.am has '.' after 'mpi' so libmpi.la won't
exist at the time libmpi_f77.la needs it.  Likewise, the C++ and F90
libraries are built before libmpi.la.

Am I missing anything?  Fixing this nicely may require some fiddling
with the source tree layout.

Cheers,
Ralf

Re: [OMPI devel] Help on building openmpi with "-Wl, --as-needed -Wl, --no-undefined"

[Ralf Wildenhues]

* Funda Wang wrote on Sun, Jul 20, 2008 at 05:29:57AM CEST:
> I'm currently building openmpi 1.2.6 under Mandriva cooker, and its
> default LDFLAGS is "-Wl,--as-needed -Wl,--no-undefined".
> 
> But openmpi 1.2.6 builds failed with:
> libtool: link: g++ -shared -nostdlib
> /usr/lib/gcc/i586-manbo-linux-gnu/4.3.1/../../../crti.o
> /usr/lib/gcc/i586-manbo-linux-gnu/4.3.1/crtbeginS.o  .libs/mpicxx.o
> .libs/intercepts.o .libs/comm.o .libs/datatype.o .libs/file.o
> .libs/win.o   -lnsl -lutil -L/usr/lib/gcc/i586-manbo-linux-gnu/4.3.1
> -L/usr/lib/gcc/i586-manbo-linux-gnu/4.3.1/../../.. -lstdc++ -lm
> -lpthread -lc -lgcc_s
> /usr/lib/gcc/i586-manbo-linux-gnu/4.3.1/crtendS.o
> /usr/lib/gcc/i586-manbo-linux-gnu/4.3.1/../../../crtn.o  -march=i586
> -mtune=generic -pthread -Wl,--no-undefined   -pthread -Wl,-soname
> -Wl,libmpi_cxx.so.0 -o .libs/libmpi_cxx.so.0.0.0
> .libs/mpicxx.o: In function `Errhandler':
> /home/fwang/rpm/BUILD/openmpi-1.2.6/ompi/mpi/cxx/../../../ompi/mpi/cxx/errhandler.h:30:
> undefined reference to `ompi_mpi_errors_are_fatal'

I suppose ompi/mpi/cxx/Makefile.am is missing some libmpi_cxx_la_LIBADD
line.

Cheers,
Ralf

Re: [OMPI devel] autogen error

[Ralf Wildenhues]

Hello Leonardo,

* Leonardo Fialho wrote on Thu, Jun 19, 2008 at 04:29:30PM CEST:
> [Running] aclocal -I config
> /usr/local/bin/m4:config/mca_no_configure_components.m4:9: ERROR: end of  
> file in string
> autom4te: /usr/local/bin/m4 failed with exit status: 1
> aclocal: autom4te failed with exit status: 1

Go to the toplevel source directory.
Run
   aclocal -I config

to confirm that it's this one that is failing.
Edit config/mca_no_configure_components.m4, changing line 9,
  m4_define(mca_backtrace_m4_config_component_list, [execinfo, printstack, 
darwin, none])

into
  m4_define([mca_backtrace_m4_config_component_list], [execinfo, printstack, 
darwin, none])

then rerun
  aclocal -I config

to confirm that it's failing at line 10 or later only, now.

I assume the file is being read by m4 twice, and that exposes the
underquotation in it: it should be
  m4_define([name], [value])

because otherwise, the second time, 'name' will be expanded before
m4_define is called.  Jeff, please fix this in autogen.sh.

If I'm wrong, then I'd like to know
  /usr/local/bin/m4 --version

Cheers,
Ralf

Re: [OMPI devel] libdir not propagated to contrib/vt/vt ??

[Ralf Wildenhues]

Hello,

* David Daniel wrote on Wed, Jun 11, 2008 at 11:34:23PM CEST:
> Building against recent heads (r18643) it appears that libdir (as set  
> by ./configure --prefix=$PREFIX --libdir=$PREFIX/lib64 for example) is  
> not propagated to ompi-trunk/contrib/vt/vt.

ompi/contrib/vt/configure.m4 is responsible for this.  It seems only
selected arguments from the top-level configure are propagated.  This
looks unfortunate to me, as there are many arguments the user may change
(see configure --help).

> Feature or bug?

Somebody else will have to answer that.  FWIW, I'd regard it as a bug.

Cheers,
Ralf

Re: [OMPI devel] some Makefile.am questions

[Ralf Wildenhues]

* Jeff Squyres wrote on Fri, Jun 06, 2008 at 03:21:03AM CEST:
> On Jun 4, 2008, at 5:15 PM, Ralf Wildenhues wrote:
> > 1) This is from test/Makefile.am:
> >
> > --- snip ---
> > # This should be libsupport.a, not libsupport.la.  Automake doesn't
> > # support check_LTLIBRARIES, as technically you have to install a
> > # shared library before you can use it.
> > #
> > check_LIBRARIES = libsupport.a

> > The statement in the comment is not true; Automake supports
> > check_LTLIBRARIES, and they don't have to be installed before use
> > either.  What may be confusing is that, by default, check_LTLIBRARIES
> > will be convenience archives rather than shared libraries.  If you  
> > want
> > to have an uninstalled shared library for testing, then you can use
> >
> > check_LTLIBRARIES = libsupport.la
> > # induce libtool to create a shared library:
> > libsupport_la_LDFLAGS = -rpath /nowhere
> > libsupport_la_SOURCES = \
> >...
> 
> Ah, good to know.  Other than the comment being wrong, is this a  
> problem?

I don't think so.

> > 2) test/dss/ has only a Makefile with 'CC = mpicc' hardcoded.  That
> > looks like it won't use the correct (uninstalled) mpicc but requires a
> > prior 'make install'.  Not sure whether that's intentional.
> 
> It is actually; these tests are always manually created by a  
> developer.  Our "make check" framework is not very complete.  Maybe  
> someday...

Ah, ok.  I somehow thought you must have completed this long ago,
having a test framework for uninstalled programs and libraries.

Cheers,
Ralf

Re: [OMPI devel] parallel make install

[Ralf Wildenhues]

Hello Ralph,

* Ralph Castain wrote on Tue, Jun 03, 2008 at 11:59:07PM CEST:
> Very interesting. Don't know if it's the same problem, but I noted an issue
> quite a while ago where make -jN all/install would fail when traversing
> opal. I built a workaround that was just a script that does make all in
> opal, then goes back to make -jN for orte/ompi.

Well if it doesn't, then it'd be helpful to see a build log showing the
failure.

Cheers,
Ralf

Re: [OMPI devel] parallel make install

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Tue, Jun 03, 2008 at 11:11:32PM CEST:
> ERROR: Command returned a non-zero exist status
>make -j 4 distcheck
[...]
> Making install in etc
> make[3]: Entering directory 
> `/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_build/opal/etc'
> make[4]: Entering directory 
> `/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_build/opal/etc'
> test -z 
> "/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_inst/etc"
>  || /bin/mkdir -p 
> "/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_inst/etc"
> /usr/bin/install -c -m 644 ../../../opal/etc/openmpi-mca-params.conf 
> /home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_inst/etc/openmpi-mca-params.conf
> /usr/bin/install: cannot create regular file 
> `/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_inst/etc/openmpi-mca-params.conf':
>  No such file or directory
> make[4]: *** [install-data-local] Error 1
> make[4]: *** Waiting for unfinished jobs
> make[4]: *** Waiting for unfinished jobs
> make[4]: Leaving directory 
> `/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_build/opal/etc'
> make[3]: *** [install-am] Error 2
> make[3]: Leaving directory 
> `/home/mpiteam/openmpi/nightly-tarball-build-root/trunk/create-r18551/ompi/openmpi-1.3a1r18551/_build/opal/etc'
> make[2]: *** [install-recursive] Error 1

Nice clue, thanks.  This is a bug in opal/etc/Makefile.am:

--- quote opal/etc/Makefile.am ---
# This has to be here, even though it's empty, so that AM thinks that
# "something" will happen here (details fuzzy, but we remember that this
# *needs* to be here -- you have been warned).

sysconf_DATA = 

# Steal a little trickery from a generated Makefile to only install
# files if they do not already exist at the target.

install-data-local:
@ p="$(opal_config_files)"; \
for file in $$p; do \
  if test -f $(DESTDIR)$(sysconfdir)/$$file; then \
echo "*** WARNING 
"; \
echo "*** Not installing new $$file over existing file in:"; \
echo "***   $(DESTDIR)$(sysconfdir)/$$file"; \
echo "*** WARNING 
"; \
  else \
if test -f "$$file"; then d=; else d="$(srcdir)/"; fi; \
f="`echo $$file | sed -e 's|^.*/||'`"; \
echo " $(INSTALL_DATA) $$d$$file $(DESTDIR)$(sysconfdir)/$$f"; \
$(INSTALL_DATA) $$d$$file $(DESTDIR)$(sysconfdir)/$$f; \
  fi; \
done
--- snip ---

To clarify the mysterious comment above, the "sysconf_DATA =" line
causes automake to emit an undocumented target install-sysconfDATA which
effectively runs something like
  mkdir -p $(DESTDIR)$(sysconfdir)

and then installs zero files there.  The install-data-local rule is also
updated as a dependency of 'install', just like install-sysconfDATA,
however there exists no dependency relation between the two.  Which
means that with parallel make, they can be run concurrently, which I
assume is what happened in your case; although the log shows them in the
right order, it can still happen that mkdir wasn't done with its work
before install-data-local accessed the directory.

An easy fix is to use install-data-hook instead, which is documented to
run after the normal install rules; or to generate the directory in the
install-data-local rule itself, and drop the sysconf_DATA line.

Proposed, untested patch below.

I have not checked whether there are more instances of this in OMPI.

Cheers,
Ralf

Fix race condition in 'make install': let install-data-local
create $(sysconfdir), rather than an automake-generated rule
which may be run in parallel (with make -j).

Index: opal/etc/Makefile.am
===
--- opal/etc/Makefile.am(Revision 17766)
+++ opal/etc/Makefile.am(Arbeitskopie)
@@ -23,16 +23,11 @@

 EXTRA_DIST = $(opal_config_files)

-# This has to be here, even though it's empty, so that AM thinks that
-# "something" will happen here (details fuzzy, but we remember that this
-# *needs* to be here -- you have been warned).
-
-sysconf_DATA = 
-
 # Steal a little trickery from a generated Makefile to only install
 # files if they do not already exist at the target.

 install-data-local:
+   $(mkdir_p) $(DESTDIR)$(sysconfdir)
@ p="$(opal_config_files)"; \
for file in $$p; do \
  if test -f $(DESTDIR)$(sysconfdir)/$$file; then \

[OMPI devel] parallel make install

[Ralf Wildenhues]

Hello there,

a change recently mentions this:

| r18552 | timattox | 2008-05-30 06:39:48 +0200 (Fri, 30 May 2008) | 4 lines
| Changed paths:
|M /trunk/contrib/nightly/create_tarball.sh
| 
| Apparently "make -j 4 distcheck" has a race condition when "installing" in
| parallel.  Remove the "-j 4" so we don't get random tarball build failures.
| Hopefully this won't take all that much longer to make the tarball each night.


Can you fill me in on details here, like the nature of the race or a
build log showing the failure, the Automake version used to autogen.sh,
and so on?  Because if that turns out to be a race within Automake code,
it'd be nice to fix it; well, and if it turns out to be a race in OMPI
code, it'd be nice to fix it, too.  ;-)

Thanks!
Ralf

Re: [OMPI devel] r18551 - brakes ob1 compilation on Sles10

[Ralf Wildenhues]

* Pavel Shamis (Pasha) wrote on Mon, Jun 02, 2008 at 02:25:13PM CEST:
> r18551 brakes ompi compilation on SLES10 gcc 4.1.0.
>
> I got follow error on my systems  
> (http://www.open-mpi.org/mtt/index.php?do_redir=672 ):

[...]
> /usr/lib64/gcc/x86_64-suse-linux/4.1.0/../../../../x86_64-suse-linux/bin/ld: 
> .libs/pml_ob1_sendreq.o: relocation R_X86_64_PC32 against  
> `mca_pml_ob1_rndv_completion' can not be used when making a shared  
> object; recompile with -fPIC

The build log shows that your clock isn't set properly, so I'd first fix
that and do a complete rebuild afterwards.  The log also shows that
.libs/pml_ob1_sendreq.o is compiled with -fPIC, so second I'd assume a
compiler or binutils bug.  The GCC bugzilla lists
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30153
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28781
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21382
as possible starting points for further investigation.  Maybe your
distributor has fixed or newer binutils packages for you.

Cheers,
Ralf

Re: [OMPI devel] Fix for XLC + libtool issue

[Ralf Wildenhues]

* Jeff Squyres wrote on Fri, Apr 25, 2008 at 01:54:39PM CEST:
> On Apr 25, 2008, at 7:40 AM, Ralf Wildenhues wrote:
> 
> > <http://thread.gmane.org/gmane.comp.sysutils.autoconf.patches/3985>
> Wow -- those timings are impressive!  Quoting that URL (OMPI is [1]):
> 
> -
> For example[1], in a large package with 871 substituted variables, of  
> which 2*136 are produced by AM_CONDITIONAL, and roughly 210 Makefiles.  
> './config.status' execution for those Makefiles (no headers, no  
> depfiles):
> - with Automake-1.9.6: 78.54user 9.32system 1:38.60elapsed 89%CPU  
> (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major 
> +2551217minor)pagefaults 0swaps
> - with Automake 1.10 (no superfluous $(*_TRUE)/$(*_FALSE) settings):  
> 56.11user 8.31system 1:16.51elapsed 84%CPU (0avgtext+0avgdata  
> 0maxresident)k 0inputs+0outputs (0major+2284709minor)pagefaults 0swaps
> - additionally with the Autoconf patch below: 11.24user 3.62system  
> 0:21.89elapsed 67%CPU (0avgtext+0avgdata 0maxresident)k 0inputs 
> +0outputs (0major+935332minor)pagefaults 0swaps
> -
> 
> Is the "with the Autoconf patch below" equivalent to AM 1.10 + AC 2.62?

Yes.  The patch from the message made it into Autoconf 2.62.  OMPI is a
poster child in hitting the quadratic overhead with Autoconf 2.59.

Cheers,
Ralf

[OMPI devel] 3 test failures

[Ralf Wildenhues]

Hello,

I've just stumbled over three testsuite failures on GNU/Linux x86,
with an out-of-tree build (mkdir build; cd build;
../ompi_trunk/configure -C).  Hope I'm not completely off-topic here...

Cheers,
Ralf

PASS: ompi_bitmap
--
Sorry!  You were supposed to get help about:
opal_init:startup:internal-failure
from the file:
help-opal-runtime.txt
But I couldn't find any file matching that name.  Sorry!
--
 Failure : Comparison failure
 Expected result: 0
 Test result: -13
SUPPORT: OMPI Test failed: opal_hash_table_t (1 of 1 failed)
FAIL: opal_hash_table
--
Sorry!  You were supposed to get help about:
opal_init:startup:internal-failure
from the file:
help-opal-runtime.txt
But I couldn't find any file matching that name.  Sorry!
--
 Failure : Comparison failure
 Expected result: 0
 Test result: -13
SUPPORT: OMPI Test failed: (null) (1 of 1 failed)
FAIL: opal_list
--
Sorry!  You were supposed to get help about:
opal_init:startup:internal-failure
from the file:
help-opal-runtime.txt
But I couldn't find any file matching that name.  Sorry!
--
 Failure : Comparison failure
 Expected result: 0
 Test result: -13
SUPPORT: OMPI Test failed: opal_value_array_t (1 of 1 failed)
FAIL: opal_value_array

[OMPI devel] getting config.guess/config.sub from upstream

[Ralf Wildenhues]

Hello,

Please note that the CVS repo for config.guess and config.sub is
outdated, development has moved to use git.
ompi_trunk/config/distscript.csh could be adjusted to pull from

and likewise for config.sub.  I'm too dumb to fix the csh script though,
for me it seems to always fail the download (it did that before, too).

Cheers,
Ralf

Re: [OMPI devel] more vt woes

[Ralf Wildenhues]

Hallo Matthias,

* Matthias Jurenz wrote on Wed, Feb 13, 2008 at 01:49:41PM CET:
> On Di, 2008-02-12 at 11:27 -0500, George Bosilca wrote:
> 
> > I keep getting some warnings when I compile with gcc-4.2 on MAC OS X.
> > 
> > tools/compwrap/Makefile.am:38: `CXXFLAGS' is a user variable, you  
> > should not override it;
[...]
> So, please ignore the warnings from Automake... Currently, I see no
> better solution ;-)

You can put
  AUTOMAKE_OPTIONS = -Wno-gnu

in tools/compwrap/Makefile.am to avoid the warnings from automake.

Cheers,
Ralf

[OMPI devel] Fixlet for config/ompi_contrib.m4

[Ralf Wildenhues]

Hello,

please apply this patch, to make future contrib integration just a tad
bit easier.  I verified that the generated configure script is
identical, minus whitespace and comments.

Cheers,
Ralf

2008-02-11  Ralf Wildenhues  <ralf.wildenh...@gmx.de>

* config/ompi_contrib.m4 (OMPI_CONTRIB): Unify listings of
contrib software packages.

Index: config/ompi_contrib.m4
===
--- config/ompi_contrib.m4  (Revision 17419)
+++ config/ompi_contrib.m4  (Arbeitskopie)
@@ -67,20 +67,13 @@
 # Cycle through each of the hard-coded software packages and
 # configure them if not disabled.  May someday be expanded to have
 # autogen find the packages instead of this hard-coded list
-# (https://svn.open-mpi.org/trac/ompi/ticket/1162).  I couldn't
-# figure out a simple/easy way to have the m4 foreach do the m4
-# include *and* all the rest of the stuff, so I settled for having
-# two lists: each contribted software package will need to add its
-# configure.m4 list here and then add its name to the m4 define
-# for contrib_software_list.  Cope.
-#dnlm4_include(ompi/contrib/libnbc/configure.m4)
-m4_include(ompi/contrib/vt/configure.m4)
-
-m4_define(contrib_software_list, [vt])
-#dnlm4_define(contrib_software_list, [libnbc, vt])
+# (https://svn.open-mpi.org/trac/ompi/ticket/1162).
+# m4_define([contrib_software_list], [libnbc, vt])
+m4_define([contrib_software_list], [vt])
 m4_foreach(software, [contrib_software_list],
-   [OMPI_CONTRIB_DIST_SUBDIRS="$OMPI_CONTRIB_DIST_SUBDIRS 
contrib/software"
-   _OMPI_CONTRIB_CONFIGURE(software)])
+  [m4_include([ompi/contrib/]software[/configure.m4])
+  OMPI_CONTRIB_DIST_SUBDIRS="$OMPI_CONTRIB_DIST_SUBDIRS 
contrib/software"
+  _OMPI_CONTRIB_CONFIGURE(software)])

 # Setup the top-level glue
 AC_SUBST(OMPI_CONTRIB_SUBDIRS)

Re: [OMPI devel] VT integration: make distclean problem

[Ralf Wildenhues]

* Josh Hursey wrote on Mon, Feb 11, 2008 at 07:31:25PM CET:
> I've been noticing another problem with the VT integration. If you do  
> a "./configure --enable-contrib-no-build=vt" a subsequent 'make  
> distclean' will fail in contrib/vt. The 'make distclean' will succeed  
> with VT enabled (default).

ATM the toplevel configury does not run configure in contrib/vt/vt, if
that is disabled.  I think that's intended.  But it also means that a
distribution built from such a build tree cannot be complete, i.e.,
contain all contribs, because their Makefiles do not exist which contain
the respective dist rules.  Likewise for distclean and maintainer-clean.

I suppose for distclean, this could be worked around uglily (please
speak up if you want me to take a shot at it), but if you want all of
these to work out of the box even for --enable-contrib-no-build=vt, then
you need to run configure for vt every time.  Sorry 'bout that.

Cheers,
Ralf

Re: [OMPI devel] More VT warnings

[Ralf Wildenhues]

* Tim Prins wrote on Fri, Feb 01, 2008 at 04:09:31PM CET:
> 
> Note that this indicates that the file vt_metric_papi.c is being 
> compiled *3* times. I am not using a parallel make here. Any ideas why 
> it is compiling 3 times?

The file is listed as source file to four different libraries, and
per-target CFLAGS are used for these.  Between one and four of these
libraries are actually built, depending on decisions done at configure
time.

Cheers,
Ralf

Re: [OMPI devel] vt compiler warnings and errors

[Ralf Wildenhues]

* Jeff Squyres wrote on Thu, Jan 31, 2008 at 07:10:36PM CET:
> Ah -- I didn't notice this before -- do you have a configure script  
> committed to SVN?  If so, this could be the problem.

> > On Do, 2008-01-31 at 08:09 -0500, Tim Prins wrote:
[...]
> >> [tprins@sif test]$ make clean
> >> 
> >> Making clean in otf
> >> make[5]: Entering directory
> >> `/san/homedirs/tprins/sif/test/ompi/contrib/vt/vt/extlib/otf'
> >>   cd . && /bin/sh
> >> /u/tprins/sif/test/ompi/contrib/vt/vt/extlib/otf/missing --run
> >> automake-1.10 --gnu
> >> cd . && /bin/sh /u/tprins/sif/test/ompi/contrib/vt/vt/extlib/otf/ 
> >> missing
> >> --run autoconf
[...]

These files do not belong in SVN, they are generated by aclocal:
  ompi/contrib/vt/vt/extlib/otf/aclocal.m4
  ompi/contrib/vt/vt/aclocal.m4

Cheers,
Ralf

Re: [OMPI devel] Fwd: === CREATE FAILURE ===

[Ralf Wildenhues]

Hello,

* Brian W. Barrett wrote on Thu, Jan 24, 2008 at 09:43:12PM CET:
> Automake forces v7 mode so that Solaris tar can untar the tarball, IIRC.

You can choose how it should behave:



Cheers,
Ralf

Re: [OMPI devel] configure patch

[Ralf Wildenhues]

* Jeff Squyres wrote on Sat, Jan 19, 2008 at 02:03:02AM CET:
> On Jan 8, 2008, at 6:53 PM, Elan Ruusamäe wrote:
> >
> > please consider applying this patch
> > http://cvs.pld-linux.org/cgi-bin/cvsweb.cgi/SOURCES/openmpi-ksh.patch
> >
> > fixes build process when /bin/sh is not bash (for example pdksh)

> This patch seems to work for me on Linux.
> 
> Sun -- can you test this with the Solaris sh?

This patch will not work with Solaris sh.  Please consider using this
instead, it should avoid the issues with both shells.  (Untested.)

Cheers,
Ralf


Index: config/ompi_get_version.m4
===
--- config/ompi_get_version.m4  (Revision 17165)
+++ config/ompi_get_version.m4  (Arbeitskopie)
@@ -41,7 +41,7 @@

 dnl quote eval to suppress macro expansion with non-GNU m4
 if test -f "$1"; then
-[eval] "`sed -n \"\
+ompi_vers=`sed -n "
t clear
: clear
s/^major/$2_MAJOR_VERSION/
@@ -53,7 +53,8 @@
t print
b
: print
-   p\" < \"\$1\"`"
+   p" < "$1"`
+   [eval] "$ompi_vers"

 # Only print release version if it isn't 0
 if test $$2_RELEASE_VERSION -ne 0 ; then

Re: [OMPI devel] small configure cache variable bug

[Ralf Wildenhues]

* Ralf Wildenhues wrote on Thu, Oct 11, 2007 at 11:36:53PM CEST:
> 
>   * config/ompi_check_visibility.m4 (OMPI_CHECK_VISIBILITY):
>   Rename ompi_cv_cc_fvisibility to ompi_vc_cc_fvisibility, so
>   that it will be cached.

Of course then I get it wrong myself.  I meant to write:

* config/ompi_check_visibility.m4 (OMPI_CHECK_VISIBILITY):
Rename ompi_vc_cc_fvisibility to ompi_cv_cc_fvisibility, so
that it will be cached.

Sorry about that.

[OMPI devel] small configure cache variable bug

[Ralf Wildenhues]

Hello Open MPI Developers,

Here's a small patch to fix a misnamed cache variable.  The next
Autoconf version will issue a warning for typos like these, which
is how I found this.

Cheers,
Ralf

* config/ompi_check_visibility.m4 (OMPI_CHECK_VISIBILITY):
Rename ompi_cv_cc_fvisibility to ompi_vc_cc_fvisibility, so
that it will be cached.

Index: config/ompi_check_visibility.m4
===
--- config/ompi_check_visibility.m4 (revision 16430)
+++ config/ompi_check_visibility.m4 (working copy)
@@ -33,7 +33,7 @@
 CFLAGS="$CFLAGS_orig -fvisibility=hidden"
 add=
 AC_CACHE_CHECK([if $CC supports -fvisibility],
-[ompi_vc_cc_fvisibility],
+[ompi_cv_cc_fvisibility],
 [AC_TRY_LINK([
 #include 
 __attribute__((visibility("default"))) int foo;
@@ -42,17 +42,17 @@
 [if test -s conftest.err ; then
 $GREP -iq "visibility" conftest.err
 if test "$?" = "0" ; then
-ompi_vc_cc_fvisibility="no"
+ompi_cv_cc_fvisibility="no"
 else
-ompi_vc_cc_fvisibility="yes"
+ompi_cv_cc_fvisibility="yes"
 fi
  else
-ompi_vc_cc_fvisibility="yes"
+ompi_cv_cc_fvisibility="yes"
  fi],
-[ompi_vc_cc_fvisibility="no"])
+[ompi_cv_cc_fvisibility="no"])
 ])

-if test "$ompi_vc_cc_fvisibility" = "yes" ; then
+if test "$ompi_cv_cc_fvisibility" = "yes" ; then
 add=" -fvisibility=hidden"
 have_visibility=1
 AC_MSG_WARN([$add has been added to CFLAGS])

Re: [OMPI devel] FreeBSD Support?

[Ralf Wildenhues]

Hello Karol,

* Karol Mroz wrote on Wed, Sep 19, 2007 at 07:23:50PM CEST:
> When running the autogen.sh script as non-root, I see the following error:
[...]
>   autom4te-2.61: cannot open configure: Permission denied
[...]
> After some searching, it would appear that this is an autoconf issue
> that crops up in FreeBSD but, for whatever reason, not in Linux. A quick
> workaround is to add: `chmod -vr u+w *` just before autogen issues the
> `run_and_check $ompi_autoconf` command on line 438.

I can look into this if needed.  Is it sufficient to make
opal/libltdl/configure writable, or its containing directory?

As a workaround you should be able to use nightly tarballs
instead of the SVN version.

Cheers,
Ralf

Re: [OMPI devel] [RFC] Upgrade to newer libtool 2.1 snapshot

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Fri, Aug 03, 2007 at 10:33:51PM CEST:
> WHAT: Upgrade to a newer Libtool 2.1 nightly snapshot (we are  
> currently using 1.2362 2007/01/23) for making OMPI tarballs.
> 
> WHY: https://svn.open-mpi.org/trac/ompi/ticket/982 is fixed by newer  
> Libtool snapshots (e.g., 1.2444 2007/04/10 is what I have installed  
> at Cisco).

Is it?  If so, then I would like to know why (config.log outputs for
both would be nice).  Could have been an Autoconf update instead.
Asking because I don't think the bug was consciously fixed in Libtool;
only a test was added to expose the issue.  I'll put it on my list of
things to look at.

> Plus, it's a newer version, so it's better, right?  ;-)

FWIW, a patch applied today fixes a regression introduced on 2007-05-08
and reported by Brian.

Cheers,
Ralf

Re: [OMPI devel] Multi-environment builds

[Ralf Wildenhues]

* Jeff Squyres wrote on Tue, Jul 10, 2007 at 01:28:40PM CEST:
> On Jul 10, 2007, at 2:42 AM, Ralf Wildenhues wrote:
> 
> >> 1. The most obvious one (to me, at least) is to require that  
> >> people provide
> >> "--with-xx" when they build the system.
> >
> > I'll throw in another one for good measure: If --with-xx is given,
> > build with the component.  If --without-xx is given, disable it.
> > If neither is given, do as you currently do: enable it if you find
> > suitable libraries.
> 
> FWIW, we have this already:
[...]

Ah good, I must confess I wasn't aware of this existing functionality
(I don't need it myself).  Thanks.

> > In case the number of components gets too large, have a switch to
> > turn off automatic discovery even in the nonpresence of --with* flags.
> 
> Did you mean the equivalent of the --enable-mca-no-build switch, or  
> disable *all* automatic discovery?

I meant: disable all automatic discovery.  But you guys are in a much
better position to decide which way is more useful.  All I wanted is for
you to be aware of existing possibilities (which you are, but that
wasn't obvious to me).

Cheers,
Ralf

Re: [OMPI devel] Multi-environment builds

[Ralf Wildenhues]

Hello Ralph,

* Ralph Castain wrote on Tue, Jul 10, 2007 at 03:51:06AM CEST:
> 
> The problem is that our Open MPI build system automatically detects the
> presence of those libraries, builds the corresponding components, and then
> links those libraries into our system. Unfortunately, this causes two
> side-effects:
[...]
> A couple of solutions come immediately to mind:
> 
> 1. The most obvious one (to me, at least) is to require that people provide
> "--with-xx" when they build the system.

I'll throw in another one for good measure: If --with-xx is given,
build with the component.  If --without-xx is given, disable it.
If neither is given, do as you currently do: enable it if you find
suitable libraries.

In case the number of components gets too large, have a switch to
turn off automatic discovery even in the nonpresence of --with* flags.

It may be a bit more work on the Open MPI configury, but it may be
more convenient for your users.

2 cents from somebody who's not going to have to implement it.  ;-)

Cheers,
Ralf

Re: [OMPI devel] Build system changes

[Ralf Wildenhues]

* Ralph Castain wrote on Thu, Nov 30, 2006 at 04:12:16PM CET:
> That could be the problem. I had to update automake, and unfortunately
> Darwin Ports hasn't reached that level yet. So I had to build and install
> automake by hand under my own bin directory. But libtool etc are all in
> /usr/bin.

Then that's the most likely cause.

* Ralph Castain wrote on Thu, Nov 30, 2006 at 04:10:35PM CET:
> m4 version is 1.4.2

Updating is advisable.  M4 1.4.8 is the current version.  It has
many bugfixes over 1.4.2, and there are a couple of tricky cases
where the latter is known to fail (in conjunction with autoconf).

If you excuse a small plug: Automake 1.10 and (unreleased) Autoconf 2.62
will both help to eliminate a large part of config.status execution time,
compared with the 1.9.6/2.59 combo (OpenMPI was used for testing here):
http://lists.gnu.org/archive/html/autoconf-patches/2006-11/msg00035.html

Cheers,
Ralf

Re: [OMPI devel] Build system changes

[Ralf Wildenhues]

> * Ralph Castain wrote on Thu, Nov 30, 2006 at 03:55:05PM CET:
> > 
> > configure.in:1994: warning: macro `AM_PROG_LIBTOOL' not found in library
> > [Running] autoheader
> > [Running] autoconf
> > configure.in:1997: error: possibly undefined macro: AM_PROG_LIBTOOL
> >   If this token and others are legitimate, please use m4_pattern_allow.
> >   See the Autoconf documentation.

Another question: does the aclocal from Automake 1.10 find the
libtool.m4 file from your Libtool installation?  IOW, if Libtool
and Automake are not installed below the same $prefix, then you
need to do something: either put the path to libtool.m4 in the
file $automake_prefix/share/aclocal/dirlist, or pass some -I
parameters to aclocal.

Hope that helps.

Cheers,
Ralf

Re: [OMPI devel] Build system changes

[Ralf Wildenhues]

* Ralph Castain wrote on Thu, Nov 30, 2006 at 03:55:05PM CET:
> 
> I updated this morning and hit a few problems. I found I was using autoconf
> 2.60, so I had to update to automake 1.10 as you indicated. However, once I
> did that, I still couldn't build due to the following errors:
> 
> configure.in:1994: warning: macro `AM_PROG_LIBTOOL' not found in library
> [Running] autoheader
> [Running] autoconf
> configure.in:1997: error: possibly undefined macro: AM_PROG_LIBTOOL
>   If this token and others are legitimate, please use m4_pattern_allow.
>   See the Autoconf documentation.

> I am running:
> 
> Autoconf 2.60
> Automake 1.10
> Libtool 1.5.22

Which version of m4 do you use?  Did libtoolize throw an error (earlier
in the autogen output)?

Cheers,
Ralf

[OMPI devel] help config.status to not mess up substitutions

[Ralf Wildenhues]

Please apply this robustness patch, which helps to avoid accidental
unwanted substitutions done by config.status.  From all I can tell,
they do not happen now, but first the Autoconf manual warns against
them, second they make some config.status optimizations so much more
difficult than necessary.  :-)

In unrelated news, I tested Automake 1.10 with OpenMPI, and it saves
about 15s of config.status time, and about half a minute of `make dist'
time on my system.  Some pending Fortran changes have only made it into
Automake after 1.10 was released.

Cheers,
Ralf

2006-10-23  Ralf Wildenhues  <ralf.wildenh...@gmx.de>

* opal/tools/wrappers/Makefile.am: Protect manual substitutions
from config.status.
* ompi/tools/wrappers/Makefile.am: Likewise.
* orte/tools/wrappers/Makefile.am: Likewise.

Index: opal/tools/wrappers/Makefile.am
===
--- opal/tools/wrappers/Makefile.am (revision 12254)
+++ opal/tools/wrappers/Makefile.am (working copy)
@@ -76,8 +76,8 @@

 opalcc.1: opal_wrapper.1
rm -f opalcc.1
-   sed -e 's/@COMMAND@/opalcc/g' -e 's/@PROJECT@/Open PAL/g' -e 
's/@PROJECT_SHORT@/OPAL/g' -e 's/@LANGUAGE@/C/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > opalcc.1
+   sed -e 's/[@]COMMAND[@]/opalcc/g' -e 's/[@]PROJECT[@]/Open PAL/g' -e 
's/[@]PROJECT_SHORT[@]/OPAL/g' -e 's/[@]LANGUAGE[@]/C/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > opalcc.1

 opalc++.1: opal_wrapper.1
rm -f opalc++.1
-   sed -e 's/@COMMAND@/opalc++/g' -e 's/@PROJECT@/Open PAL/g' -e 
's/@PROJECT_SHORT@/OPAL/g' -e 's/@LANGUAGE@/C++/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > opalc++.1
+   sed -e 's/[@]COMMAND[@]/opalc++/g' -e 's/[@]PROJECT[@]/Open PAL/g' -e 
's/[@]PROJECT_SHORT[@]/OPAL/g' -e 's/[@]LANGUAGE[@]/C++/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > opalc++.1
Index: ompi/tools/wrappers/Makefile.am
===
--- ompi/tools/wrappers/Makefile.am (revision 12254)
+++ ompi/tools/wrappers/Makefile.am (working copy)
@@ -84,20 +84,20 @@

 mpicc.1: $(top_srcdir)/opal/tools/wrappers/opal_wrapper.1
rm -f mpicc.1
-   sed -e 's/@COMMAND@/mpicc/g' -e 's/@PROJECT@/Open MPI/g' -e 
's/@PROJECT_SHORT@/OMPI/g' -e 's/@LANGUAGE@/C/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpicc.1
+   sed -e 's/[@]COMMAND[@]/mpicc/g' -e 's/[@]PROJECT[@]/Open MPI/g' -e 
's/[@]PROJECT_SHORT[@]/OMPI/g' -e 's/[@]LANGUAGE[@]/C/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpicc.1

 mpic++.1: $(top_srcdir)/opal/tools/wrappers/opal_wrapper.1
rm -f mpic++.1
-   sed -e 's/@COMMAND@/mpic++/g' -e 's/@PROJECT@/Open MPI/g' -e 
's/@PROJECT_SHORT@/OMPI/g' -e 's/@LANGUAGE@/C++/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpic++.1
+   sed -e 's/[@]COMMAND[@]/mpic++/g' -e 's/[@]PROJECT[@]/Open MPI/g' -e 
's/[@]PROJECT_SHORT[@]/OMPI/g' -e 's/[@]LANGUAGE[@]/C++/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpic++.1

 mpicxx.1: $(top_srcdir)/opal/tools/wrappers/opal_wrapper.1
rm -f mpicxx.1
-   sed -e 's/@COMMAND@/mpicxx/g' -e 's/@PROJECT@/Open MPI/g' -e 
's/@PROJECT_SHORT@/OMPI/g' -e 's/@LANGUAGE@/C++/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpicxx.1
+   sed -e 's/[@]COMMAND[@]/mpicxx/g' -e 's/[@]PROJECT[@]/Open MPI/g' -e 
's/[@]PROJECT_SHORT[@]/OMPI/g' -e 's/[@]LANGUAGE[@]/C++/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpicxx.1

 mpif77.1: $(top_srcdir)/opal/tools/wrappers/opal_wrapper.1
rm -f mpif77.1
-   sed -e 's/@COMMAND@/mpif77/g' -e 's/@PROJECT@/Open MPI/g' -e 
's/@PROJECT_SHORT@/OMPI/g' -e 's/@LANGUAGE@/Fortran 77/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpif77.1
+   sed -e 's/[@]COMMAND[@]/mpif77/g' -e 's/[@]PROJECT[@]/Open MPI/g' -e 
's/[@]PROJECT_SHORT[@]/OMPI/g' -e 's/[@]LANGUAGE[@]/Fortran 77/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpif77.1

 mpif90.1: $(top_srcdir)/opal/tools/wrappers/opal_wrapper.1
rm -f mpif90.1
-   sed -e 's/@COMMAND@/mpif90/g' -e 's/@PROJECT@/Open MPI/g' -e 
's/@PROJECT_SHORT@/OMPI/g' -e 's/@LANGUAGE@/Fortran 90/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpif90.1
+   sed -e 's/[@]COMMAND[@]/mpif90/g' -e 's/[@]PROJECT[@]/Open MPI/g' -e 
's/[@]PROJECT_SHORT[@]/OMPI/g' -e 's/[@]LANGUAGE[@]/Fortran 90/g' < 
$(top_srcdir)/opal/tools/wrappers/opal_wrapper.1 > mpif90.1
Index: orte/tools/wrappers/Makefile.am
===
--- orte/tools/wrappers/Makefile.am (revision 12254)
+++ orte/tools/wrappers/Makefile.am (working copy)
@@ -51,8 +51,8 @@

 ortecc.1: $(top_srcdir)/opal/tools/wrappers/opal_wrapper.1
rm -f ortecc.1
-   sed -e 's/@COMMAND@/ortecc/g' -e 's/@PROJ

Re: [OMPI devel] Issues with 1.2 and intel/portland compilers

[Ralf Wildenhues]

Hello Orion,

* Orion Poplawski wrote on Thu, Oct 12, 2006 at 06:57:04PM CEST:
> I've been trying to build the openmpi 1.2 branch with the Intel and 
> Portland Fortran compilers and was having trouble using their 
> -i-static/-Bstatic_pgi options.

Your report touches several issues:

> I'm not sure what the most cross platform way to return only the first 
> argument is.

ompi_setup_f77.m4 should allow for more than one word in $F77.
Autoconf typically uses something like this to extract the first word:

# Extract the first word of "$ac_prog", so it can be a program name with args.
set dummy $ac_prog; ac_word=$2

so OpenMPI could use that, too (with variable names changed, of
course, so they don't interfere with Autoconf's),

> Of course, if there was a way to get these passed to the appropriate 
> link stages, we could avoid this.  But it looks like libtool strips the 
> -i-static argument and doesn't pass it on to the link command, not sure 
> about the -Bstatic_pgi argument.

Yes, this is a problem.  On the libtool command lines, you can use
  -Wc,-i-static

to get -i-static passed to the compiler driver; and
  -Wl,-i-static

to get it passed to the linker directly (but that would be wrong in this
case).  Unfortunately, you may not in general be able to stuff this in
LDFLAGS at configure time (I haven't tried, it may just happen to work),
so one different way would be to stick it in at make time only:
  make LDFLAGS=...

But then, configuring with 
> FC="ifort -i-static"

is a much better option: it tests the right setting at configure time.

Note there may still be issues: libtool should actually be able to
understand what -i-static really means (and for example not worry about
those libraries that are linked statically).  But the FC setting should
get you going.

Hope that helps.

Cheers,
Ralf

Re: [OMPI devel] [Open MPI] #334: Building with Libtool 2.1a fails to run OpenIB BTL

[Ralf Wildenhues]

Hello,

* Open MPI wrote on Wed, Sep 06, 2006 at 01:00:00PM CEST:
> #334: Building with Libtool 2.1a fails to run OpenIB BTL

>Are you testing uninstalled or installed programs/libraries?
> 
>  Installed.
> 
>If you are testing an uninstalled program: does libtool generate a shell
>  wrapper for it?  If yes: post the two shell wrappers generated by 1.5.22
>  and HEAD.
> 
>  I was testing with a trivial "hello world" MPI application; i.e., one that
>  I had compiled with mpicc and was running with "mpirun -np 2 --mca btl
>  openib hello".  Hence, I was testing against installed trees of Open MPI.
>  I took care to "rm -rf" the installation tree before testing each so that
>  there would be no kruft left from prior installs.

OK, I can only assume that with 1.5.22, some code links against
libibverbs, or loads it earlier at runtime, so that the symbol is
present.  In any case I wonder why mthca.so isn't linked directly
against libibverbs (maybe useful to suggest that to upstream).

To find out the culprit, here's a couple of quick (well) ways:
  diff build-with-lt1.5/Makefile build-with-lt2.1a/Makefile

and look for differences in library link variables.  Otherwise, the
config.log outputs should give clues.

To give you some more hints for possible causes: ompi_info could have a
different set of RPATH entries, or different NEEDED libraries than your
test executable; if any of those cause libibverbs.so to be loaded, then
the symbol would be visible already.  Maybe your test executables even
have different RPATHs or NEEDED libs (find out with 'objdump -p' and
ldd)?

>  I've attached 2 tarballs to the bug (you have to go to the URL of the bug
>  to get them; they are not included in the mails):

If there are tarballs available at
http://svn.open-mpi.org/trac/ompi/ticket/334, then I'm too blind to find
them.  Would that be elsewhere?

>   * One with all the configure output and the wrapper script for ompi_info.
>  Note that ompi_info -- which lt_dlopen()'s the OpenIB BTL -- does not show
>  the same problem (i.e., the OpenIB BTL opens properly and ompi_info shows
>  its information).  This happens with both the uninstalled and installed
>  ompi_info.
>   * Another with the same for 2.1a.

That would be very helpful.

Cheers,
Ralf

Re: [OMPI devel] OpenMPI not conforming with the C90 spec?

[Ralf Wildenhues]

* Ralph H Castain wrote on Mon, Aug 21, 2006 at 02:39:51PM CEST:
>
> It sounds, therefore, like we are now C99 compliant and no longer C90
> compliant at all?

Well, a compiler supporting C90 plus 'long long' as an extension would
still be ok.  Surely, that's not "strictly C90".  But from glancing at
the mpi.h file in my build tree, some declarations are commented out if
HAVE_LONG_LONG is not set.  Your comments indicate that things still
would not work with a strict C90 compiler.

> I don't know how big a deal that is, but if true it is something possibly
> worth noting on our web site and in our release notes. The code will
> definitely NOT compile unless int64_t is defined and supported.

I think for Solaris 2.5.1 and Tru64 4.0, you would need a replacement
definition, but I guess those systems aren't targets for OpenMPI either.

> PS to Ralf: actually, quite a few systems exist today that do not support
> long long or int64_t. The majority of computers in the world, in fact, do
> not do so - they are in embedded systems.

Right.  None of them is fully C99 compliant.  AFAIK, some allow (slow!)
software emulations for long long types.

> We decided that we were tasked
> with supporting high-performance computing systems instead, and those are
> now built almost exclusively from systems that DO support such structures.
> Just a point of correctness... :-)

Sure.  For practical matters, I would always go for C99 + extensions (as
in gcc's -std=gnu99; you could use AC_PROG_CC_STDC from Autoconf-2.60
for some sane choices).

Cheers,
Ralf

[OMPI devel] exit declaration in configure tests

[Ralf Wildenhues]

Revision 11268 makes me curious:

|M /trunk/config/ompi_setup_cxx.m4
| 
| Reorder the C++ compiler discovery stages. Check first the compiler vendor
| before checking if we are able to compile the test program. This is required
| for windows as the C++ conftest.c file generated by configure cannot be
| compiled with the Microsoft cl.exe compiler (because of the exit function
| prototype). So if we detect a vendor equal to microsoft we will assume
| that the compiler is correctly installed (which is true on Windows most
| of the time anyway).

I believe to have killed all problematic exit cases from the OpenMPI
configury some time ago.  Did I miss any, or did the remaining ones
come from some other package (so we can fix that one)?  Which Autoconf
version was used (2.60 should not use any exit declarations itself any
more)?

Cheers,
Ralf

Re: [OMPI devel] OpenMPI not conforming with the C90 spec?

[Ralf Wildenhues]

Hello Adrian, Jonathan,

* Adrian Knoth wrote on Sat, Aug 19, 2006 at 08:38:15PM CEST:
> On Thu, Aug 17, 2006 at 11:48:44PM +0100, Jonathan Underwood wrote:
>  
> > Compiling a file with the gcc options -Wall and -pedantic gives the
> > following warning:
> > mpi.h:147: warning: ISO C90 does not support 'long long'
> > Is this intentional, or is this a bug?
> 
> If you do not insist on using C90, you may compile with -std=c99
> to get rid of this message ;)

More precisely, the OpenMPI installation that provides this mpi.h file
was compiled with 'long long' support, which quite undoubtedly is a
feature rather than a bug.  Users should not compile their code with
strict C89 settings.

> I don't have the C90 (ANSI-C) at my fingertips, but I confirm it
> does not support "long long".

Yes.

> Perhaps we should use int64_t instead.

No, that would not help: int64_t is C99, so it should not be declared
either in C89 mode.  Also, the int64_t is required to have 64 bits, and
could thus theoretically be smaller than 'long long' (no, I don't think
any such systems exist today).

Cheers,
Ralf

[OMPI devel] robustify config against picky compiler flags

[Ralf Wildenhues]

Autoconf has recently seen some effort to robustify configure tests
against picky GCC warning settings.  There is quite a way to go yet, but
it may still be useful to move the OpenMPI configury in that direction,
too.  So below is a patch that picks some low-hanging fruit on the way
to make
  ../ompi-trunk/configure -C CFLAGS='-W -Wall -Werror' \
 CXXFLAGS='-W -Wall -Werror'

work better with GCC.  Note that currently, the above will still produce
some wrong results, or may even fail one some systems; so it's really
just a first step in that direction.

Also please double-check the changes to Windows specific files -- I did
not test them in any way.

As a side effect, this patch fixes one typo in a comment, and one
underquoted pair of brackets that would make code from c_weak_symbols.m4
end up as
  int main(int argc, char *argv);

in the configure script.

Cheers,
Ralf

Index: config/ompi_check_pthread_pids.m4
===
--- config/ompi_check_pthread_pids.m4   (revision 11225)
+++ config/ompi_check_pthread_pids.m4   (working copy)
@@ -44,7 +44,7 @@
 #include 
 #include 
 void *checkpid(void *arg);
-int main(int argc, char* argv[]) {
+int main() {
   pthread_t thr;
   int pid, retval;
   pid = getpid();
Index: config/ompi_microsoft.m4
===
--- config/ompi_microsoft.m4(revision 11225)
+++ config/ompi_microsoft.m4(working copy)
@@ -39,7 +39,7 @@

 AC_MSG_CHECKING(for working InterlockedCompareExchange)
 AC_TRY_RUN( [#include 
- int main( int argc, char** argv ) {
+ int main() {
  LONG dest = 1, exchange = 0, comperand = 1;
  SetErrorMode(SEM_FAILCRITICALERRORS);
  InterlockedCompareExchange( , exchange, comperand );
@@ -55,7 +55,7 @@

 AC_MSG_CHECKING(for working InterlockedCompareExchangeAcquire)
 AC_TRY_RUN( [#include 
- int main( int argc, char** argv ) {
+ int main() {
  LONG dest = 1, exchange = 0, comperand = 1;
  SetErrorMode(SEM_FAILCRITICALERRORS);
  InterlockedCompareExchangeAcquire( , exchange, 
comperand );
@@ -71,7 +71,7 @@

 AC_MSG_CHECKING(for working InterlockedCompareExchangeRelease)
 AC_TRY_RUN( [#include 
- int main( int argc, char** argv ) {
+ int main() {
  LONG dest = 1, exchange = 0, comperand = 1;
  SetErrorMode(SEM_FAILCRITICALERRORS);
  InterlockedCompareExchangeRelease( , exchange, 
comperand );
@@ -87,7 +87,7 @@

 AC_MSG_CHECKING(for working InterlockedCompareExchange64)
 AC_TRY_RUN( [#include 
- int main( int argc, char** argv ) {
+ int main() {
  LONGLONG dest = 1, exchange = 0, comperand = 1;
  SetErrorMode(SEM_FAILCRITICALERRORS);
  InterlockedCompareExchange64( , exchange, comperand 
);
Index: config/ompi_setup_libevent.m4
===
--- config/ompi_setup_libevent.m4   (revision 11225)
+++ config/ompi_setup_libevent.m4   (working copy)
@@ -225,7 +225,7 @@
 #include 

 int
-main(int argc, char **argv)
+main()
 {
int kq;
int n;
@@ -289,7 +289,7 @@
 }

 int
-main(int argc, char **argv)
+main()
 {
int epfd;

Index: config/f77_get_value_true.m4
===
--- config/f77_get_value_true.m4(revision 11225)
+++ config/f77_get_value_true.m4(working copy)
@@ -56,7 +56,6 @@

 void $ompi_print_logical_fn(ompi_fortran_logical_t * logical)
 {
-int result = 0;
 FILE *f=fopen("conftestval", "w");
 if (!f) exit(1);

@@ -89,7 +88,7 @@
  [happy=0])

  if test "$happy" = "0" ; then
- AC_MSG_ERROR([Could not determine value of Fotran .TRUE..  
Aborting.])
+ AC_MSG_ERROR([Could not determine value of Fortran .TRUE..  
Aborting.])
  fi

  AS_IF([test "$cross_compiling" = "yes"],
Index: config/ompi_config_asm.m4
===
--- config/ompi_config_asm.m4   (revision 11225)
+++ config/ompi_config_asm.m4   (working copy)
@@ -211,7 +211,7 @@
 }
 #endif
 int
-main(int argc, char *argv[[]])
+main()
 {
 gsym_test_func();
 return 0;
Index: config/cxx_find_template_repository.m4
===
--- config/cxx_find_template_repository.m4  (revision 11225)
+++ config/cxx_find_template_repository.m4  (working copy)
@@ -91,7 +91,7 @@
 }

 int
-main(int argc, char *argv[])
+main()
 {
   foo var1(6);
   foo< foo > var2(var1);
Index: 

[OMPI devel] tests/datatype: srandomdev

[Ralf Wildenhues]

Hello everyone,

Currently 'make check' is failing for me in the tests/datatype
directory, because my system (GNU/Linux, glibc 2.3.6) doesn't provide a
srandomdev function, called in checksum.c.  I believe it's BSD specific;
if you need it, I guess you could add a configure time check for it.

Cheers, and keep up the good work,
Ralf

[OMPI devel] some configury update

[Ralf Wildenhues]

The following patches fix two rather nasty issues with configury, and
one missing bit of GCS update.  All changes should be backwards
compatible (in the sense that they will work with older Autoconf
versions), the first two are necessary for correct functioning with
Autoconf-2.59 already, but the first will really bite you when you
eventually use 2.60, as it will cause a configure script with syntax
errors.  The third is to support the directory names change mandated by
a GNU Coding Standards change (this change was actually found by a
warning measure Autoconf applies to configure scripts during the move).

The patches are against the trunk, but I guess similar ones apply to
whatever OpenMPI branch runs danger of being bootstrapped with 2.60
eventually.

Cheers,
Ralf

* config/ompi_config_asm.m4 (OMPI_CHECK_POWERPC_REG): Fix M4
quoting.
* ompi/mca/io/romio/romio/configure.in: Fix some more M4
quoting.

* config/ompi_config_subdir.m4 (OMPI_CONFIG_SUBDIR):
More consistent quoting, a la CVS Autoconf.
* config/ompi_config_subdir_args.m4 (OMPI_CONFIG_SUBDIR_ARGS):
The (undocumented!) Autoconf variable $ac_configure_args needs
to be evaluated, to account for the quoting done.

* ompi/mca/io/romio/romio/util/romioinstall.in: Set datarootdir,
necessary for Autoconf-2.60 which will define some variables
based upon this value (e.g., datadir, docdir).

Index: config/ompi_config_asm.m4
===
--- config/ompi_config_asm.m4   (revision 10115)
+++ config/ompi_config_asm.m4   (working copy)
@@ -424,10 +424,10 @@
 OMPI_TRY_ASSEMBLE([$ompi_cv_asm_text
 addi 1,1,0],
 [ompi_cv_asm_powerpc_r_reg=0],
-OMPI_TRY_ASSEMBLE([$ompi_cv_asm_text
+[OMPI_TRY_ASSEMBLE([$ompi_cv_asm_text
 addi r1,r1,0],
 [ompi_cv_asm_powerpc_r_reg=1],
-AC_MSG_ERROR([Can not determine how to use PPC registers])))
+[AC_MSG_ERROR([Can not determine how to use PPC registers])])])
 if test "$ompi_cv_asm_powerpc_r_reg" = "1" ; then
 AC_MSG_RESULT([yes])
 else
Index: config/ompi_config_subdir.m4
===
--- config/ompi_config_subdir.m4(revision 10115)
+++ config/ompi_config_subdir.m4(working copy)
@@ -131,8 +131,8 @@
 export LDFLAGS LIBS
 sub_configure="$SHELL '$subdir_srcdir/configure'"
 AC_MSG_NOTICE([running $sub_configure $subdir_args 
--cache-file=$subdir_cache_file --srcdir=$subdir_srcdir])
-eval $sub_configure $subdir_args \
-   --cache-file=$subdir_cache_file --srcdir=$subdir_srcdir
+eval "$sub_configure $subdir_args \
+   --cache-file=\"\$subdir_cache_file\" --srcdir=\"$subdir_srcdir\""
 if test "$?" = "0"; then
eval $subdir_success
AC_MSG_NOTICE([$sub_configure succeeded for $subdir_dir])
Index: config/ompi_config_subdir_args.m4
===
--- config/ompi_config_subdir_args.m4   (revision 10115)
+++ config/ompi_config_subdir_args.m4   (working copy)
@@ -33,7 +33,10 @@
 subdirs_args=
 subdirs_skip=no

-for subdirs_arg in $ac_configure_args; do
+eval "set x $ac_configure_args"
+shift
+for subdirs_arg
+do
 if test "$subdirs_skip" = "yes"; then
subdirs_skip=no
 else
@@ -51,7 +54,10 @@
-srcdir=* | --srcdir=*)
;;
*) 
-   subdirs_args="$subdirs_args $subdirs_arg" 
+   case $subdir_arg in
+   *\'*) subdir_arg=`echo "$subdir_arg | sed "s/'/'''/g"` ;;
+   esac
+   subdirs_args="$subdirs_args '$subdirs_arg'" 
;;
esac
 fi
@@ -61,8 +67,8 @@
 # Assign the output
 #

-subdirs_str="$1="'"'"$subdirs_args"'"'
-eval $subdirs_str
+subdirs_str=$1=\"$subdirs_args\"
+eval "$subdirs_str"

 #
 # Clean up
Index: ompi/mca/io/romio/romio/configure.in
===
--- ompi/mca/io/romio/romio/configure.in(revision 10118)
+++ ompi/mca/io/romio/romio/configure.in(working copy)
@@ -1475,7 +1475,7 @@
 #
 AC_CHECK_HEADERS(sys/stat.h sys/types.h unistd.h)
 AC_CHECK_FUNCS(stat,
-AC_DEFINE(HAVE_STAT, 1, Define if stat function is present)
+[AC_DEFINE(HAVE_STAT, 1, Define if stat function is present)
 AC_MSG_CHECKING([for st_fstype member of stat structure])
 AC_TRY_COMPILE([
#ifdef HAVE_SYS_TYPES_H
@@ -1496,14 +1496,14 @@
AC_DEFINE(ROMIO_HAVE_STRUCT_STAT_WITH_ST_FSTYPE, 1, Define if struct 
stat has a st_fstype member),
AC_MSG_RESULT(no)
 )
-)
+])

 #
 # Check for statvfs and f_basetype field (Solaris, Irix, AIX, etc.)
 #
 AC_CHECK_HEADERS(sys/types.h sys/statvfs.h sys/vfs.h)
 AC_CHECK_FUNCS(statvfs,
-AC_DEFINE(HAVE_STATVFS, 1, Define if statvfs function is present)
+[AC_DEFINE(HAVE_STATVFS, 1, Define if statvfs function is 

Re: [OMPI devel] bug in Makefile.in

[Ralf Wildenhues]

* Dries Kimpe wrote on Mon, May 08, 2006 at 08:46:49PM CEST:
> 
> 2) The $ in grep 'pvfs$' was added by me as a temporary hack to
> prevent it from building pvfs along with pvfs2.
>   It's not a good solution anyway, because $FILE_SYSTEM contains things
>   as "nfs ufs pvfs pvfs2"
> 
> Really, the correct fix is just to use $file_system_pvfs,
> $file_system_pvfs2, ...

Nah.  Don't use two processes per conditional if you can get away with
none.  Let's not make the build slower than necessary.

> Dries Kimpe wrote:
> > AM_CONDITIONAL(BUILD_GRIDFTP, [test -n "`echo $FILE_SYSTEM | grep 
> > gridftp`"])

AM_CONDITIONAL(BUILD_GRIDFTP,
  [{ case " $FILE_SYSTEM " in " gridftp ") :;; *) false;; esac; }])

If you don't want to type each, you could use something along these
lines:

m4_foreach([my_fs], [hfs, nfs, panfs, ...],
  [AM_CONDITIONAL(BUILD_[]AS_TR_CPP(my_fs),
[{ case " $FILE_SYSTEM " in \ my_fs\ ) :;; *) false;; esac; }])
  ])

Cheers,
Ralf

Re: [OMPI devel] typo in ompi/mca/io/romio/configure.m4?

[Ralf Wildenhues]

Hi Brian, Dries,

* Brian Barrett wrote on Mon, May 08, 2006 at 01:43:38PM CEST:
> 
> However, we really *should* abort configure and warn the user if -- 
> enable-io-romio is given and ROMIO fails to configure.  I will fix  
> this with the LDFLAGS fix.  I'll let you know when the fixes become  
> available

While you're looking at config/ompi_config_subdir* anyway, you may be
interested to know that it mistreats some arguments with special
characters in it.  That may not be so important to OpenMPI, but when
passing the same config.cache file down to the sub-configure (which
OpenMPI does not do), then it can be:
http://lists.gnu.org/archive/html/bug-autoconf/2006-04/msg00087.html
http://lists.gnu.org/archive/html/autoconf-patches/2006-04/msg00357.html

It may even be that this comment from configure.ac:
# AC_CONFIG_SUBDIRS appears to be broken for non-gcc compilers (i.e.,
# passing precious variables down to the sub-configure).

hints at the very problem linked to above, but frankly, I don't know
that.  Again, if that was a different issue with AC_CONFIG_SUBDIRS,
it'd be great to hear about it.

Cheers,
Ralf