Re: [OMPI users] Open MPI programs with autoconf/automake?
Hi Jeff, Jeff Squyres wrote: On Nov 10, 2008, at 6:41 AM, Jed Brown wrote: With #define's and compiler flags, I think that can be easily done -- was wondering if this is something that developers using MPI do and whether AC/AM supports it. AC will allow you to #define whatever you want -- look at the documentation for AC_DEFINE and AC_DEFINE_UNQUOTED. You can also tell your configure script to accept various --with- and --enable- arguments; see the docs for AC_ARG_WITH and AC_ARG_ENABLE. Thanks for this! I know "it's in the document", but I've been going through it with much difficulty. Definitely complete, but hard to get into and know what it is I need. So, some keywords to search for will definitely help! If --with-mpi is not specified, the following will happen: 1. You don't set CC (and friends), so AC_PROG_CC will find the default compilers. Hence, your app will not be compiled and linked against the MPI libraries. 2. #define BUILDING_WITH_MPI to 0, so the code above will compile out the call to MPI_Send(). Both of these are valid techniques -- use whichever suits your app the best. I see; thank you for giving me this second option. I guess I'm more attracted to this since it allows me to continue working with Open MPI. As I am writing the system [now], I'll have to keep in mind to make it modular so that parts can be #define'd in and out easily. Thank you for your careful explanation! Ray
Re: [OMPI users] Open MPI programs with autoconf/automake?
On Nov 10, 2008, at 6:41 AM, Jed Brown wrote: With #define's and compiler flags, I think that can be easily done -- was wondering if this is something that developers using MPI do and whether AC/AM supports it. AC will allow you to #define whatever you want -- look at the documentation for AC_DEFINE and AC_DEFINE_UNQUOTED. You can also tell your configure script to accept various --with- and --enable- arguments; see the docs for AC_ARG_WITH and AC_ARG_ENABLE. The normal way to do this is by building against a serial implementation of MPI. Lots of parallel numerical libraries bundle such an implementation so you could just grab one of those. For example, see PETSc's mpiuni ($PETSC_DIR/include/mpiuni/mpi.h and $PETSC_DIR/src/sys/mpiuni/mpi.c) which implements many MPI calls as macros. Note that your serial implementation only needs to provide the subset of MPI that your program actually uses. For instance, if you never send messages to yourself, you can implement MPI_Send as MPI_Abort since it should never be called in serial. This is one viable way to do it. Another way that I have seen is to use #define's (via AC_DEFINE / AC_DEFINE_UNQUOTED) to both define BUILDING_WITH_MPI to 0 or 1 (or some variant) and conditionally use the MPI wrapper compilers (or not) to build your application. This technique is best suited to applications have are highly modular and can easily segregate all your calls to MPI in a single area that can be turned on / off with a #define. To put this more concretely, you can have this: ./configure --with-mpi that does two things: 1. set CC=mpicc (and friends) before calling AC_PROG_CC (and friends). This will setup your app to be compiled with the MPI wrapper compilers, and therefore automatically link in libmpi, etc. 2. #define BUILDING_WITH_MPI to 1, so in your code you can do stuff like #if BUILDING_WITH_MPI MPI_Send(...); #endif If --with-mpi is not specified, the following will happen: 1. You don't set CC (and friends), so AC_PROG_CC will find the default compilers. Hence, your app will not be compiled and linked against the MPI libraries. 2. #define BUILDING_WITH_MPI to 0, so the code above will compile out the call to MPI_Send(). Both of these are valid techniques -- use whichever suits your app the best. -- Jeff Squyres Cisco Systems
Re: [OMPI users] Open MPI programs with autoconf/automake?
Hi Nuno,
Thank you for the link and your update to it. I definitely don't mind
that it isn't "pretty"! :-)
Since your post, I've been trying to understand it and how to work it
in. But, I think I've been making some progress over the weekend.
Thank you!
Ray
Nuno Sucena Almeida wrote:
Hi,
see if this macro solves your problem:
http://autoconf-archive.cryp.to/acx_mpi.html
it requires some improvement, but might be a start. Since I only have OpenMPI
I use it in the following way(it's not pretty):
configure.ac:
(...)
dnl Check for MPI
dnl This check will set the MPICC and MPICXX variables to the MPI compiler
ones
dnl if the library is found, or to the regular compilers if not.
AC_ARG_WITH(mpi, [AC_HELP_STRING([--with-mpi],
[enable MPI support [default=yes]])],
[case "${withval}" in
yes|no) with_mpi=$withval;;
*)
AC_MSG_ERROR(bad value ${withval} for --with-mpi);;
esac],
[with_mpi=yes])
if test "x$with_mpi" = "xyes"; then
ACX_MPI([], [AC_MSG_ERROR(could not find mpi library for --with-mpi)])
AC_DEFINE(HAVE_MPI)
MPI_CXXLIBS=`mpicxx --showme:link`
MPI_CXXFLAGS=`mpicxx --showme:compile`
AC_SUBST(MPI_CXXLIBS)
AC_SUBST(MPI_CXXFLAGS)
else
MPICC="$CC"
MPICXX="$CXX"
AC_SUBST(MPICC)
AC_SUBST(MPICXX)
fi
AM_CONDITIONAL([WE_HAVE_MPI],[test "x$with_mpi" = "xyes"])
(...)
Makefile.am:
(...)
# MPI headers/libraries:
INCLUDES+=$(MPI_CXXFLAGS)
OTHERLIBS+=$(MPI_CXXLIBS)
etc
I would start by improving the mentioned macro with specific support for each
MPI implementation...
Nuno
On Thursday 06 November 2008 06:35:33 am Raymond Wan wrote:
I'm not sure if this is relevant to this mailing list, but I'm trying to
get autoconf/automake working with an Open MPI program I am writing (in
Re: [OMPI users] Open MPI programs with autoconf/automake?
Hi Jeff, Thank you for your reply! Indeed, I was never going to look at OMPI's use of AC/AM...no doubt that would be far too complex for me. :-) The AC/AM documents I have found so far are quite difficult for me -- I really am starting from zero. Prior to using MPI, I have been writing my own Makefiles and for the projects I work on [usually alone], that was sufficient. However, your tip did help me; I dropped "MPI" from my search and added "tutorial" instead and the hits are better. As a starting point, I probably will only support open source MPIs. One thing I was wondering about was whether it is possible, though the use of #define's, to create code that is both multi-processor (MPI/mpic++) and single-processor (normal g++). That is, if users do not have any MPI installed, it compiles it with g++. With #define's and compiler flags, I think that can be easily done -- was wondering if this is something that developers using MPI do and whether AC/AM supports it. But thanks and I'll try more phrase combinations in google... Ray Jeff Squyres wrote: OMPI itself uses AC/AM to build itself, but our configure.ac and some of our Makefile.am's are fairly complex -- I wouldn't use these as starting points. You probably want to start with some general AC/AM tutorials (the AM documentation reads somewhat like a tutorial -- you might want to look there?). Just google around for AC/AM tutorials; leave "MPI" out of your searching. Indeed, all you really want to do is build your software -- the only real difference between your app and other apps is that you want to use mpicc and friends to build it (vs. gcc and friends). Most other aspects should be the same. Hence, the big difference for building an MPI application with AC/AM is that you want to set the C, C++, and Fortran compilers to the various "wrapper" MPI compilers (e.g., CC=mpicc, CXX=mpic++, FC=mpif90, F77=mpif77). Then AC_PROG_CC (etc.) will find the wrapper compiler instead of gcc (for example). It gets tricky, though, because not all MPI implementations have wrapper compilers -- so it's up to you to decide how portable you want to be. The open source MPI's both have wrapper compilers by the same names (mpicc et al.), but some of the vendor/MPP platform-specific MPI's may not. Good luck. On Nov 6, 2008, at 6:35 AM, Raymond Wan wrote: Hi all, I'm not sure if this is relevant to this mailing list, but I'm trying to get autoconf/automake working with an Open MPI program I am writing (in C++) but unfortunately, I don't know where to begin. I'm new to both tools but have it working well enough for a non-MPI program. When I google for these terms, I end up with results of people who have problems with autoconf/automake and *installing* Open MPI -- which isn't what I am looking for. Or, I get results that are well beyond what I need...I just need something to start with and won't be combining programming languages, etc. Does anyone have a brief example of configure.ac and/or Makefile.am to start me off or know of a tutorial that describes how they can be adapted for Open MPI from a non-MPI program? Thank you -- any help appreciated! Ray ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] Open MPI programs with autoconf/automake?
Hi,
see if this macro solves your problem:
http://autoconf-archive.cryp.to/acx_mpi.html
it requires some improvement, but might be a start. Since I only have
OpenMPI
I use it in the following way(it's not pretty):
configure.ac:
(...)
dnl Check for MPI
dnl This check will set the MPICC and MPICXX variables to the MPI compiler
ones
dnl if the library is found, or to the regular compilers if not.
AC_ARG_WITH(mpi, [AC_HELP_STRING([--with-mpi],
[enable MPI support [default=yes]])],
[case "${withval}" in
yes|no) with_mpi=$withval;;
*)
AC_MSG_ERROR(bad value ${withval} for --with-mpi);;
esac],
[with_mpi=yes])
if test "x$with_mpi" = "xyes"; then
ACX_MPI([], [AC_MSG_ERROR(could not find mpi library for --with-mpi)])
AC_DEFINE(HAVE_MPI)
MPI_CXXLIBS=`mpicxx --showme:link`
MPI_CXXFLAGS=`mpicxx --showme:compile`
AC_SUBST(MPI_CXXLIBS)
AC_SUBST(MPI_CXXFLAGS)
else
MPICC="$CC"
MPICXX="$CXX"
AC_SUBST(MPICC)
AC_SUBST(MPICXX)
fi
AM_CONDITIONAL([WE_HAVE_MPI],[test "x$with_mpi" = "xyes"])
(...)
Makefile.am:
(...)
# MPI headers/libraries:
INCLUDES+=$(MPI_CXXFLAGS)
OTHERLIBS+=$(MPI_CXXLIBS)
etc
I would start by improving the mentioned macro with specific support for each
MPI implementation...
Nuno
On Thursday 06 November 2008 06:35:33 am Raymond Wan wrote:
> I'm not sure if this is relevant to this mailing list, but I'm trying to
> get autoconf/automake working with an Open MPI program I am writing (in
--
http://aeminium.org/slug/
Re: [OMPI users] Open MPI programs with autoconf/automake?
OMPI itself uses AC/AM to build itself, but our configure.ac and some of our Makefile.am's are fairly complex -- I wouldn't use these as starting points. You probably want to start with some general AC/AM tutorials (the AM documentation reads somewhat like a tutorial -- you might want to look there?). Just google around for AC/AM tutorials; leave "MPI" out of your searching. Indeed, all you really want to do is build your software -- the only real difference between your app and other apps is that you want to use mpicc and friends to build it (vs. gcc and friends). Most other aspects should be the same. Hence, the big difference for building an MPI application with AC/AM is that you want to set the C, C++, and Fortran compilers to the various "wrapper" MPI compilers (e.g., CC=mpicc, CXX=mpic++, FC=mpif90, F77=mpif77). Then AC_PROG_CC (etc.) will find the wrapper compiler instead of gcc (for example). It gets tricky, though, because not all MPI implementations have wrapper compilers -- so it's up to you to decide how portable you want to be. The open source MPI's both have wrapper compilers by the same names (mpicc et al.), but some of the vendor/MPP platform-specific MPI's may not. Good luck. On Nov 6, 2008, at 6:35 AM, Raymond Wan wrote: Hi all, I'm not sure if this is relevant to this mailing list, but I'm trying to get autoconf/automake working with an Open MPI program I am writing (in C++) but unfortunately, I don't know where to begin. I'm new to both tools but have it working well enough for a non-MPI program. When I google for these terms, I end up with results of people who have problems with autoconf/automake and *installing* Open MPI -- which isn't what I am looking for. Or, I get results that are well beyond what I need...I just need something to start with and won't be combining programming languages, etc. Does anyone have a brief example of configure.ac and/or Makefile.am to start me off or know of a tutorial that describes how they can be adapted for Open MPI from a non-MPI program? Thank you -- any help appreciated! Ray ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres Cisco Systems
