Re: [OMPI users] check for CUDA support
Thanks for the clarification, Jeff.
Andrei,
MPIX_Query_cuda_support indeed just returns the value of OPAL_CUDA_SUPPORT
which is set to one if configure included --with-cuda.
/* If CUDA-aware support is configured in, return 1. Otherwise, return 0.
* This API may be extended to return more features in the future. */
int MPIX_Query_cuda_support(void)
{
return OPAL_CUDA_SUPPORT;
}
You can see the details of how OPAL_CUDA_SUPPORT is set in
root_dir/config/opal_check_cuda.m4
In summary, you could use MPIX_Query_cuda_support in your application to
check during runtime or you could check ompi_info to see if the build was
configured correctly --with-cuda before running a cuda-aware MPI
application.
On Tue, Oct 30, 2018 at 1:42 PM Jeff Squyres (jsquyres)
wrote:
> Oh, I should clarify: that MPIX_Query_cuda_support() function is always
> present in Open MPI -- even if you don't compile with CUDA support.
>
> That's kinda the point / it's probably obvious, but I thought I would
> clarify, anyway. :-)
>
>
> > On Oct 30, 2018, at 4:29 PM, Akshay Venkatesh
> wrote:
> >
> > +1 to what Jeff said.
> >
> > So you would need --with-cuda pointing to a cuda installation to have
> cuda-awareness in OpenMPI.
> >
> > On Tue, Oct 30, 2018 at 12:47 PM Jeff Squyres (jsquyres) via users <
> users@lists.open-mpi.org> wrote:
> > The "Configure command line" shows you the command line that was given
> to "configure" when building Open MPI.
> >
> > The "MPI extensions" line just indicates which Open MPI "extensions"
> were built.
> >
> > CUDA is one of the possible extensions that can get built.
> >
> > The CUDA Open MPI extension is actually an API call that will tell you
> if your Open MPI has CUDA support:
> >
> > https://www.open-mpi.org/doc/v3.1/man3/MPIX_Query_cuda_support.3.php
> >
> >
> >
> >
> > > On Oct 30, 2018, at 3:14 PM, Akshay Venkatesh
> wrote:
> > >
> > > The first one is the critical one. If the build was not configured
> --with-cuda, then it's unlikely cuda-aware MPI is supported.
> > >
> > > On Tue, Oct 30, 2018, 12:10 PM Andrei Berceanu <
> andreicberce...@gmail.com> wrote:
> > > Thanks! I tried both and
> > > ompi_info -a | grep "\-with\-cuda" returns nothing, while
> > > ompi_info -a | grep "xtensions" returns
> > > MPI extensions: affinity, cuda
> > >
> > > It seems the two outputs are in conflict, what does that mean?
> > >
> > >
> > > On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh <
> akshay.v.3...@gmail.com> wrote:
> > > Andrei,
> > >
> > > I generally check with one of these two:
> > >
> > > $ ompi_info -a | grep "\-with\-cuda"
> > > Configure command line: '--prefix=$HOME/ompi/build-cuda'
> '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda'
> '--with-ucx=$HOME/ucx-github/build'
> '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug'
> '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem'
> '--enable-install-libpmix' '--with-ompi-pmix-rte'
> > >
> > > $ ompi_info -a | grep "xtensions"
> > > MPI extensions: affinity, cuda, pcollreq
> > >
> > >
> > > On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu <
> andreicberce...@gmail.com> wrote:
> > > Hi all,
> > >
> > > I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if
> it has CUDA support or not?
> > >
> > > Best,
> > > Andrei
> > > ___
> > > users mailing list
> > > users@lists.open-mpi.org
> > > https://lists.open-mpi.org/mailman/listinfo/users
> > >
> > >
> > > --
> > > -Akshay
> > > NVIDIA
> > > ___
> > > users mailing list
> > > users@lists.open-mpi.org
> > > https://lists.open-mpi.org/mailman/listinfo/users
> > > ___
> > > users mailing list
> > > users@lists.open-mpi.org
> > > https://lists.open-mpi.org/mailman/listinfo/users
> > > ___
> > > users mailing list
> > > users@lists.open-mpi.org
> > > https://lists.open-mpi.org/mailman/listinfo/users
> >
> >
> > --
> > Jeff Squyres
> > jsquy...@cisco.com
> >
> > ___
> > users mailing list
> > users@lists.open-mpi.org
> > https://lists.open-mpi.org/mailman/listinfo/users
> >
> >
> > --
> > -Akshay
> > NVIDIA
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
>
>
--
-Akshay
NVIDIA
___
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
Oh, I should clarify: that MPIX_Query_cuda_support() function is always present in Open MPI -- even if you don't compile with CUDA support. That's kinda the point / it's probably obvious, but I thought I would clarify, anyway. :-) > On Oct 30, 2018, at 4:29 PM, Akshay Venkatesh wrote: > > +1 to what Jeff said. > > So you would need --with-cuda pointing to a cuda installation to have > cuda-awareness in OpenMPI. > > On Tue, Oct 30, 2018 at 12:47 PM Jeff Squyres (jsquyres) via users > wrote: > The "Configure command line" shows you the command line that was given to > "configure" when building Open MPI. > > The "MPI extensions" line just indicates which Open MPI "extensions" were > built. > > CUDA is one of the possible extensions that can get built. > > The CUDA Open MPI extension is actually an API call that will tell you if > your Open MPI has CUDA support: > > https://www.open-mpi.org/doc/v3.1/man3/MPIX_Query_cuda_support.3.php > > > > > > On Oct 30, 2018, at 3:14 PM, Akshay Venkatesh > > wrote: > > > > The first one is the critical one. If the build was not configured > > --with-cuda, then it's unlikely cuda-aware MPI is supported. > > > > On Tue, Oct 30, 2018, 12:10 PM Andrei Berceanu > > wrote: > > Thanks! I tried both and > > ompi_info -a | grep "\-with\-cuda" returns nothing, while > > ompi_info -a | grep "xtensions" returns > > MPI extensions: affinity, cuda > > > > It seems the two outputs are in conflict, what does that mean? > > > > > > On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh > > wrote: > > Andrei, > > > > I generally check with one of these two: > > > > $ ompi_info -a | grep "\-with\-cuda" > > Configure command line: '--prefix=$HOME/ompi/build-cuda' > > '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' > > '--with-ucx=$HOME/ucx-github/build' > > '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' > > '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' > > '--enable-install-libpmix' '--with-ompi-pmix-rte' > > > > $ ompi_info -a | grep "xtensions" > > MPI extensions: affinity, cuda, pcollreq > > > > > > On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu > > wrote: > > Hi all, > > > > I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if it > > has CUDA support or not? > > > > Best, > > Andrei > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > > > > > -- > > -Akshay > > NVIDIA > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > > > -- > -Akshay > NVIDIA -- Jeff Squyres jsquy...@cisco.com ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
+1 to what Jeff said. So you would need --with-cuda pointing to a cuda installation to have cuda-awareness in OpenMPI. On Tue, Oct 30, 2018 at 12:47 PM Jeff Squyres (jsquyres) via users < users@lists.open-mpi.org> wrote: > The "Configure command line" shows you the command line that was given to > "configure" when building Open MPI. > > The "MPI extensions" line just indicates which Open MPI "extensions" were > built. > > CUDA is one of the possible extensions that can get built. > > The CUDA Open MPI extension is actually an API call that will tell you if > your Open MPI has CUDA support: > > https://www.open-mpi.org/doc/v3.1/man3/MPIX_Query_cuda_support.3.php > > > > > > On Oct 30, 2018, at 3:14 PM, Akshay Venkatesh > wrote: > > > > The first one is the critical one. If the build was not configured > --with-cuda, then it's unlikely cuda-aware MPI is supported. > > > > On Tue, Oct 30, 2018, 12:10 PM Andrei Berceanu < > andreicberce...@gmail.com> wrote: > > Thanks! I tried both and > > ompi_info -a | grep "\-with\-cuda" returns nothing, while > > ompi_info -a | grep "xtensions" returns > > MPI extensions: affinity, cuda > > > > It seems the two outputs are in conflict, what does that mean? > > > > > > On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh < > akshay.v.3...@gmail.com> wrote: > > Andrei, > > > > I generally check with one of these two: > > > > $ ompi_info -a | grep "\-with\-cuda" > > Configure command line: '--prefix=$HOME/ompi/build-cuda' > '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' > '--with-ucx=$HOME/ucx-github/build' > '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' > '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' > '--enable-install-libpmix' '--with-ompi-pmix-rte' > > > > $ ompi_info -a | grep "xtensions" > > MPI extensions: affinity, cuda, pcollreq > > > > > > On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu < > andreicberce...@gmail.com> wrote: > > Hi all, > > > > I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if > it has CUDA support or not? > > > > Best, > > Andrei > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > > > > > -- > > -Akshay > > NVIDIA > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > ___ > > users mailing list > > users@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/users > > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > -- -Akshay NVIDIA ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
The "Configure command line" shows you the command line that was given to "configure" when building Open MPI. The "MPI extensions" line just indicates which Open MPI "extensions" were built. CUDA is one of the possible extensions that can get built. The CUDA Open MPI extension is actually an API call that will tell you if your Open MPI has CUDA support: https://www.open-mpi.org/doc/v3.1/man3/MPIX_Query_cuda_support.3.php > On Oct 30, 2018, at 3:14 PM, Akshay Venkatesh wrote: > > The first one is the critical one. If the build was not configured > --with-cuda, then it's unlikely cuda-aware MPI is supported. > > On Tue, Oct 30, 2018, 12:10 PM Andrei Berceanu > wrote: > Thanks! I tried both and > ompi_info -a | grep "\-with\-cuda" returns nothing, while > ompi_info -a | grep "xtensions" returns > MPI extensions: affinity, cuda > > It seems the two outputs are in conflict, what does that mean? > > > On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh > wrote: > Andrei, > > I generally check with one of these two: > > $ ompi_info -a | grep "\-with\-cuda" > Configure command line: '--prefix=$HOME/ompi/build-cuda' > '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' > '--with-ucx=$HOME/ucx-github/build' > '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' > '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' > '--enable-install-libpmix' '--with-ompi-pmix-rte' > > $ ompi_info -a | grep "xtensions" > MPI extensions: affinity, cuda, pcollreq > > > On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu > wrote: > Hi all, > > I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if it has > CUDA support or not? > > Best, > Andrei > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > > > -- > -Akshay > NVIDIA > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users -- Jeff Squyres jsquy...@cisco.com ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
In fact, I have "'--without-cuda": Configure command line: '--prefix=/home/andrei/src/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.3.0/openmpi-3.1.2-ocjvxwsaq3qfm6342xryaxgcesjnn4a6' '--enable-shared' '--with-wrapper-ldflags=' '--enable-static' '--without-pmi' '--enable-mpi-cxx' '--with-zlib=/home/andrei/src/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.3.0/zlib-1.2.11-m5um2ez3iajeehvt4sp7cgqnhjaybyjy' '--without-psm' '--without-psm2' '--without-verbs' '--without-mxm' '--without-ucx' '--without-libfabric' '--without-alps' '--without-lsf' '--without-tm' '--without-slurm' '--without-sge' '--without-loadleveler' '--disable-memchecker' '--with-hwloc=/home/andrei/src/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.3.0/hwloc-1.11.9-izyltij6nzgb7gmknxufz5znfg763zgq' '--disable-java' '--disable-mpi-java' '--without-cuda' '--enable-cxx-exceptions' On Tue, Oct 30, 2018 at 9:07 PM Andrei Berceanu wrote: > Thanks! I tried both and > ompi_info -a | grep "\-with\-cuda" returns nothing, while > ompi_info -a | grep "xtensions" returns > MPI extensions: affinity, cuda > > It seems the two outputs are in conflict, what does that mean? > > > On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh > wrote: > >> Andrei, >> >> I generally check with one of these two: >> >> $ ompi_info -a | grep "\-with\-cuda" >> Configure command line: '--prefix=$HOME/ompi/build-cuda' >> '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' >> '--with-ucx=$HOME/ucx-github/build' >> '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' >> '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' >> '--enable-install-libpmix' '--with-ompi-pmix-rte' >> >> $ ompi_info -a | grep "xtensions" >> MPI extensions: affinity, cuda, pcollreq >> >> >> On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu < >> andreicberce...@gmail.com> wrote: >> >>> Hi all, >>> >>> I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if >>> it has CUDA support or not? >>> >>> Best, >>> Andrei >>> ___ >>> users mailing list >>> users@lists.open-mpi.org >>> https://lists.open-mpi.org/mailman/listinfo/users >> >> >> >> -- >> -Akshay >> NVIDIA >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/users > > ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
The first one is the critical one. If the build was not configured --with-cuda, then it's unlikely cuda-aware MPI is supported. On Tue, Oct 30, 2018, 12:10 PM Andrei Berceanu wrote: > Thanks! I tried both and > ompi_info -a | grep "\-with\-cuda" returns nothing, while > ompi_info -a | grep "xtensions" returns > MPI extensions: affinity, cuda > > It seems the two outputs are in conflict, what does that mean? > > > On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh > wrote: > >> Andrei, >> >> I generally check with one of these two: >> >> $ ompi_info -a | grep "\-with\-cuda" >> Configure command line: '--prefix=$HOME/ompi/build-cuda' >> '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' >> '--with-ucx=$HOME/ucx-github/build' >> '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' >> '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' >> '--enable-install-libpmix' '--with-ompi-pmix-rte' >> >> $ ompi_info -a | grep "xtensions" >> MPI extensions: affinity, cuda, pcollreq >> >> >> On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu < >> andreicberce...@gmail.com> wrote: >> >>> Hi all, >>> >>> I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if >>> it has CUDA support or not? >>> >>> Best, >>> Andrei >>> ___ >>> users mailing list >>> users@lists.open-mpi.org >>> https://lists.open-mpi.org/mailman/listinfo/users >> >> >> >> -- >> -Akshay >> NVIDIA >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/users > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
Thanks! I tried both and ompi_info -a | grep "\-with\-cuda" returns nothing, while ompi_info -a | grep "xtensions" returns MPI extensions: affinity, cuda It seems the two outputs are in conflict, what does that mean? On Tue, Oct 30, 2018 at 8:50 PM Akshay Venkatesh wrote: > Andrei, > > I generally check with one of these two: > > $ ompi_info -a | grep "\-with\-cuda" > Configure command line: '--prefix=$HOME/ompi/build-cuda' > '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' > '--with-ucx=$HOME/ucx-github/build' > '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' > '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' > '--enable-install-libpmix' '--with-ompi-pmix-rte' > > $ ompi_info -a | grep "xtensions" > MPI extensions: affinity, cuda, pcollreq > > > On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu < > andreicberce...@gmail.com> wrote: > >> Hi all, >> >> I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if it >> has CUDA support or not? >> >> Best, >> Andrei >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/users > > > > -- > -Akshay > NVIDIA > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] check for CUDA support
Andrei, I generally check with one of these two: $ ompi_info -a | grep "\-with\-cuda" Configure command line: '--prefix=$HOME/ompi/build-cuda' '--enable-mpirun-prefix-by-default' '--with-cuda=/usr/local/cuda' '--with-ucx=$HOME/ucx-github/build' '--with-ucx-libdir=$HOME/ucx-github/build/lib' '--enable-debug' '--enable-mem-debug' '--enable-mpi-fortran=no' '--disable-oshmem' '--enable-install-libpmix' '--with-ompi-pmix-rte' $ ompi_info -a | grep "xtensions" MPI extensions: affinity, cuda, pcollreq On Tue, Oct 30, 2018 at 11:40 AM Andrei Berceanu wrote: > Hi all, > > I am using openmpi@3.1.2 on an Ubuntu@16.04.5 box, how can I check if it > has CUDA support or not? > > Best, > Andrei > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users -- -Akshay NVIDIA ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
