Re: [OMPI devel] Reminder: assign as well as request review

[Paul Hargrove]

I am so often the guy complaining about what it busted.
So, it feels nice to have contributed something *positive* on this list.

-Paul

On Fri, Jan 27, 2017 at 5:42 PM, r...@open-mpi.org  wrote:

> Thanks Paul - that does indeed help!
>
> On Jan 27, 2017, at 12:26 PM, Paul Hargrove  wrote:
>
> Ralph,
>
> It looks like GitHub *might* have rolled out the solution to your problem
> just this week:
>https://github.com/blog/2306-filter-pull-request-reviews-
> and-review-requests
>
> This appears to include an "Awaiting review from you" filter.
> Not quite a dashboard or notification, but at least a way to make the
> query.
>
> -Paul
>
>
> On Fri, Jan 27, 2017 at 7:46 AM, r...@open-mpi.org 
> wrote:
>
>> Hey folks
>>
>> Just a reminder. If you request a review from someone, GitHub doesn’t
>> show that person’s icon when looking at the list of PRs. It only shows
>> their icon and marks the PR with their ID if you actually “assign” it to
>> that person. Thus, just requesting a review without assigning the PR to
>> someone makes it impossible for them to see which PRs are awaiting their
>> attention.
>>
>> Speaking personally, I have no idea which PRs are awaiting my attention
>> unless you assign them to me. So please remember to do so.
>>
>> Thanks
>> Ralph
>>
>> ___
>> devel mailing list
>> devel@lists.open-mpi.org
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
>
>
>
> --
> Paul H. Hargrove  phhargr...@lbl.gov
> Computer Languages & Systems Software (CLaSS) Group
> Computer Science Department   Tel: +1-510-495-2352
> <(510)%20495-2352>
> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
> <(510)%20486-6900>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
>
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>



-- 
Paul H. Hargrove  phhargr...@lbl.gov
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department   Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] Reminder: assign as well as request review

[r...@open-mpi.org]

Thanks Paul - that does indeed help!

> On Jan 27, 2017, at 12:26 PM, Paul Hargrove  wrote:
> 
> Ralph,
> 
> It looks like GitHub *might* have rolled out the solution to your problem 
> just this week:
>
> https://github.com/blog/2306-filter-pull-request-reviews-and-review-requests 
> 
> 
> This appears to include an "Awaiting review from you" filter.
> Not quite a dashboard or notification, but at least a way to make the query.
> 
> -Paul
> 
> 
> On Fri, Jan 27, 2017 at 7:46 AM, r...@open-mpi.org  
> mailto:r...@open-mpi.org>> wrote:
> Hey folks
> 
> Just a reminder. If you request a review from someone, GitHub doesn’t show 
> that person’s icon when looking at the list of PRs. It only shows their icon 
> and marks the PR with their ID if you actually “assign” it to that person. 
> Thus, just requesting a review without assigning the PR to someone makes it 
> impossible for them to see which PRs are awaiting their attention.
> 
> Speaking personally, I have no idea which PRs are awaiting my attention 
> unless you assign them to me. So please remember to do so.
> 
> Thanks
> Ralph
> 
> ___
> devel mailing list
> devel@lists.open-mpi.org 
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel 
> 
> 
> 
> -- 
> Paul H. Hargrove  phhargr...@lbl.gov 
> 
> Computer Languages & Systems Software (CLaSS) Group
> Computer Science Department   Tel: +1-510-495-2352
> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] [2.0.2rc4] "make install" failure on NetBSD/i386 (libtool?)

[Paul Hargrove]

I had no problem with 2.0.2rc3 on NetBSD, but with 2.0.2rc4 I am seeing a
"make install" failure (below).
This is seen on an x86 (32-bit) platform, but not x86_64.
I cannot say for certain that this is an Open MPI regression, since there
*have* been s/w updates on this system since I last tested.

Configured with only --prefix and --disable-mpi-fortran (due to
https://github.com/open-mpi/ompi/issues/184)

-Paul

$ env LANG=C make install
[...]
Making install in mca/btl/sm
 
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/openmpi-2.0.2rc4/config/install-sh
-c -d '/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/INST/share/openmpi'
/usr/bin/install
-c -m 644
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/openmpi-2.0.2rc4/opal/mca/btl/sm/help-mpi-btl-sm.txt
'/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/INST/share/openmpi'
 
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/openmpi-2.0.2rc4/config/install-sh
-c -d '/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/INST/lib/openmpi'
 /bin/sh ../../../../libtool   --mode=install /usr/bin/install -c
mca_btl_sm.la '/home/phargrov/OMPI/openmpi
-2.0.2rc4-netbsd7-i386/INST/lib/openmpi'
libtool: warning: relinking 'mca_btl_sm.la'
libtool: install: (cd
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/BLD/opal/mca/btl/sm;
/bin/sh "/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/BLD/libtool"
 --tag CC --mode=relink gcc -std=gnu99 -O3 -DNDEBUG -finline-functions
-fno-strict-aliasing -pthread -module -avoid-version -o mca_btl_sm.la
-rpath /home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/INST/lib/openmpi
mca_btl_sm_la-btl_sm.lo mca_btl_sm_la-btl_sm_component.lo
mca_btl_sm_la-btl_sm_frag.lo
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/BLD/opal/mca/common/sm/libmca_com
mon_sm.la -lrt -lexecinfo -lm -lutil )

*** Warning: linker path does not have real file for library
-lmca_common_sm.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because I did check the linker path looking for a file starting
*** with libmca_common_sm and none of the candidates passed a file format
test
*** using a regex pattern. Last file checked:
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/BLD/opal/mca/c
ommon/sm/.libs/libmca_common_sm.so.20.0

*** Warning: libtool could not satisfy all declared inter-library
*** dependencies of module mca_btl_sm.  Therefore, libtool will create
*** a static module, that should work as long as the dlopening
*** application is linked with the -dlopen flag.
libtool: relink: ar cru .libs/mca_btl_sm.a .libs/mca_btl_sm_la-btl_sm.o
.libs/mca_btl_sm_la-btl_sm_component.o
 .libs/mca_btl_sm_la-btl_sm_frag.o
libtool: relink: ranlib .libs/mca_btl_sm.a
libtool: relink: ( cd ".libs" && rm -f "mca_btl_sm.la" && ln -s "../
mca_btl_sm.la" "mca_btl_sm.la" )
libtool: install: /usr/bin/install -c .libs/mca_btl_sm.soT
/home/phargrov/OMPI/openmpi-2.0.2rc4-netbsd7-i386/I
NST/lib/openmpi/mca_btl_sm.so
install: .libs/mca_btl_sm.soT: stat: No such file or directory
*** Error code 1

Stop.



-Paul

-- 
Paul H. Hargrove  phhargr...@lbl.gov
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department   Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] Reminder: assign as well as request review

[Paul Hargrove]

Ralph,

It looks like GitHub *might* have rolled out the solution to your problem
just this week:

https://github.com/blog/2306-filter-pull-request-reviews-and-review-requests

This appears to include an "Awaiting review from you" filter.
Not quite a dashboard or notification, but at least a way to make the query.

-Paul


On Fri, Jan 27, 2017 at 7:46 AM, r...@open-mpi.org  wrote:

> Hey folks
>
> Just a reminder. If you request a review from someone, GitHub doesn’t show
> that person’s icon when looking at the list of PRs. It only shows their
> icon and marks the PR with their ID if you actually “assign” it to that
> person. Thus, just requesting a review without assigning the PR to someone
> makes it impossible for them to see which PRs are awaiting their attention.
>
> Speaking personally, I have no idea which PRs are awaiting my attention
> unless you assign them to me. So please remember to do so.
>
> Thanks
> Ralph
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel




-- 
Paul H. Hargrove  phhargr...@lbl.gov
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department   Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] CUDA kernels in OpenMPI

[Sylvain Jeaugey]

Hi Chris,

First, you will need to have some configure stuff to detect nvcc and use 
it inside your Makefile. UTK may have some examples to show here.


For the C/C++ API, you need to add 'extern "C"' statements around the 
interfaces you want to export in C so that you can use them inside Open MPI.


You can look at the NCCL code for an example :
https://github.com/NVIDIA/nccl/blob/master/src/nccl.h#L19-L21
Note the ifdefs in case this .h is included from in C code.

In the .cu, the 'extern "C"' is burried into defines :
https://github.com/NVIDIA/nccl/blob/master/src/core.h#L149-L150

So an example would be :
myapi.h :
#ifdef __cplusplus
extern "C" {
#endif
void myfunc(...);
#ifdef __cplusplus
}
#endif
lib.cu :
extern "C" __attribute__ ((visibility("default"))) void myfunc(...) { ... }

Sylvain

On 01/27/2017 09:00 AM, Chris Ward wrote:
It looks like the mailing system deleted the attachment, so here it is 
inline

#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation.  All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation.  All rights
# reserved.
# Copyright (c) 2004-2009 High Performance Computing Center Stuttgart,
# University of Stuttgart.  All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2010  Cisco Systems, Inc.  All rights reserved.
# Copyright (c) 2012  Sandia National Laboratories. All rights 
reserved.

# Copyright (c) 2013  Los Alamos National Security, LLC. All rights
# reserved.
# Copyright (c) 2016  IBM Corporation.  All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#

AM_CPPFLAGS = $(coll_ibm_CPPFLAGS)

sources = \
   coll_ibm.h \
   coll_ibm_component.c \
   coll_ibm_module.c \
   coll_ibm_allgather.c \
   coll_ibm_allgatherv.c \
   coll_ibm_allreduce.c \
   coll_ibm_alltoall.c \
   coll_ibm_alltoallv.c \
   coll_ibm_barrier.c \
   coll_ibm_bcast.c \
   coll_ibm_exscan.c \
   coll_ibm_gather.c \
   coll_ibm_gatherv.c \
   coll_ibm_reduce.c \
   coll_ibm_reduce_scatter.c \
   coll_ibm_reduce_scatter_block.c \
   coll_ibm_scan.c \
   coll_ibm_scatter.c \
   coll_ibm_scatterv.c \
   comm_gpu.cu \
   allreduce_overlap.cc

SUFFIXES = .cu
#
#comm_gpu.lo: comm_gpu.cu
#/usr/local/cuda/bin/nvcc -gencode arch=compute_60,code=sm_60 
-lcuda -O3 --compiler-options "-O2 -fopenmp -mcpu=power8 -fPIC" -c 
comm_gpu.cu

#mv comm_gpu.o comm_gpu.lo

%.lo : %.cu
   /usr/local/cuda/bin/nvcc -gencode arch=compute_60,code=sm_60 
-lcuda -O3 --compiler-options "-O2 -fopenmp -mcpu=power8 -fPIC" -c $<

   mv $*.o .libs/
   touch $*.lo


# Make the output library in this directory, and name it either
# mca__.la (for DSO builds) or libmca__.la
# (for static builds).

if MCA_BUILD_ompi_coll_ibm_DSO
component_noinst =
component_install = mca_coll_ibm.la
else
component_noinst = libmca_coll_ibm.la
component_install =
endif

mcacomponentdir = $(ompilibdir)
mcacomponent_LTLIBRARIES = $(component_install)
mca_coll_ibm_la_SOURCES = $(sources)
if WANT_COLL_IBM_WITH_PAMI
mca_coll_ibm_la_LIBADD = $(coll_ibm_LIBS) \
 $(OMPI_TOP_BUILDDIR)/ompi/mca/common/pami/libmca_common_pami.la
else
mca_coll_ibm_la_LIBADD = $(coll_ibm_LIBS)
endif
mca_coll_ibm_la_LDFLAGS = -module -avoid-version $(coll_ibm_LDFLAGS)

noinst_LTLIBRARIES = $(component_noinst)
libmca_coll_ibm_la_SOURCES =$(sources)
libmca_coll_ibm_la_LIBADD = $(coll_ibm_LIBS)
libmca_coll_ibm_la_LDFLAGS = -module -avoid-version $(coll_ibm_LDFLAGS)

*T J (Chris) Ward, IBM Research.
Scalable Data-Centric Computing - Active Storage Fabrics - IBM System 
BlueGene

IBM United Kingdom Ltd., Hursley Park, Winchester, Hants, SO21 2JN
011-44-1962-818679
LinkedIn **_https://www.linkedin.com/profile/view?id=60628729_**
ResearchGate **_https://www.researchgate.net/profile/T_Ward2_**
**_
_**_IBM System BlueGene Research_* 
***_
_**_IBM System BlueGene Marketing_* 
*

**_
_**_IBM Resources for Global Servants_* 
***_
_**_IBM Branded Products_* ***_IBM Branded 
Swag_* ** 	




UNIX in the Cloud - Find A Place Where There's Room To Grow, with the 
original Open Standard. _Free Trial Here Today_ 

New Lamps For Old - _Diskless Remote Boot Linux_ 
from _National Center for 
High-Performance Computing, Taiwan_ 




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with 
number 741598.

Regi

[OMPI devel] CUDA kernels in OpenMPI

[Chris Ward]

It looks like the mailing system deleted the attachment, so here it is 
inline
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation.  All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation.  All rights
# reserved.
# Copyright (c) 2004-2009 High Performance Computing Center Stuttgart,
# University of Stuttgart.  All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2010  Cisco Systems, Inc.  All rights reserved.
# Copyright (c) 2012  Sandia National Laboratories. All rights 
reserved.
# Copyright (c) 2013  Los Alamos National Security, LLC. All rights
# reserved.
# Copyright (c) 2016  IBM Corporation.  All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#

AM_CPPFLAGS = $(coll_ibm_CPPFLAGS)

sources = \
coll_ibm.h \
coll_ibm_component.c \
coll_ibm_module.c \
coll_ibm_allgather.c \
coll_ibm_allgatherv.c \
coll_ibm_allreduce.c \
coll_ibm_alltoall.c \
coll_ibm_alltoallv.c \
coll_ibm_barrier.c \
coll_ibm_bcast.c \
coll_ibm_exscan.c \
coll_ibm_gather.c \
coll_ibm_gatherv.c \
coll_ibm_reduce.c \
coll_ibm_reduce_scatter.c \
coll_ibm_reduce_scatter_block.c \
coll_ibm_scan.c \
coll_ibm_scatter.c \
coll_ibm_scatterv.c \
comm_gpu.cu \
allreduce_overlap.cc

SUFFIXES = .cu
#
#comm_gpu.lo: comm_gpu.cu
#   /usr/local/cuda/bin/nvcc -gencode arch=compute_60,code=sm_60 
-lcuda -O3 --compiler-options "-O2 -fopenmp -mcpu=power8 -fPIC" -c 
comm_gpu.cu
#   mv comm_gpu.o comm_gpu.lo

%.lo : %.cu
/usr/local/cuda/bin/nvcc -gencode arch=compute_60,code=sm_60 
-lcuda -O3 --compiler-options "-O2 -fopenmp -mcpu=power8 -fPIC" -c $<
mv $*.o .libs/
touch $*.lo
 

# Make the output library in this directory, and name it either
# mca__.la (for DSO builds) or libmca__.la
# (for static builds).

if MCA_BUILD_ompi_coll_ibm_DSO
component_noinst =
component_install = mca_coll_ibm.la
else
component_noinst = libmca_coll_ibm.la
component_install =
endif

mcacomponentdir = $(ompilibdir)
mcacomponent_LTLIBRARIES = $(component_install)
mca_coll_ibm_la_SOURCES = $(sources)
if WANT_COLL_IBM_WITH_PAMI
mca_coll_ibm_la_LIBADD = $(coll_ibm_LIBS) \
$(OMPI_TOP_BUILDDIR)/ompi/mca/common/pami/libmca_common_pami.la
else
mca_coll_ibm_la_LIBADD = $(coll_ibm_LIBS)
endif
mca_coll_ibm_la_LDFLAGS = -module -avoid-version $(coll_ibm_LDFLAGS)

noinst_LTLIBRARIES = $(component_noinst)
libmca_coll_ibm_la_SOURCES =$(sources)
libmca_coll_ibm_la_LIBADD = $(coll_ibm_LIBS)
libmca_coll_ibm_la_LDFLAGS = -module -avoid-version $(coll_ibm_LDFLAGS)


T J (Chris) Ward, IBM Research. 
Scalable Data-Centric Computing - Active Storage Fabrics - IBM System 
BlueGene 
IBM United Kingdom Ltd., Hursley Park, Winchester, Hants, SO21 2JN 
011-44-1962-818679 
LinkedIn https://www.linkedin.com/profile/view?id=60628729 
ResearchGate https://www.researchgate.net/profile/T_Ward2 

IBM System BlueGene Research 
IBM System BlueGene Marketing 

IBM Resources for Global Servants 
IBM Branded Products IBM Branded Swag 


UNIX in the Cloud - Find A Place Where There's Room To Grow, with the 
original Open Standard. Free Trial Here Today 
New Lamps For Old - Diskless Remote Boot Linux from National Center for 
High-Performance Computing, Taiwan 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Problem on master

[r...@open-mpi.org]

Hello all

There is a known issue on master that we are attempting to debug. Sadly, it is 
one that only shows on multi-node operations, and the signature varies based on 
your environment. We hope to have this resolved soon (and no, it doesn’t appear 
to be due to any one specific commit).

In the interim, setting the MCA param routed=direct appears to provide a 
workaround - at least, it is working for me, and hopefully will work for you too

Ralph

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] CUDA kernels in OpenMPI

[Chris Ward]

Here is the complete Makefile so far.  I have it in directory 
ompi/mca/coll/ibm , which contains an implementation of an IBM-written 
collectives library.

It won't work as-is, because I don't know how to use 'libtool' which is 
presumably needed to do the compile. If anybody can show me a rule using 
libtool to do a compile with 'nvcc', I would be most grateful.


T J (Chris) Ward, IBM Research. 
Scalable Data-Centric Computing - Active Storage Fabrics - IBM System 
BlueGene 
IBM United Kingdom Ltd., Hursley Park, Winchester, Hants, SO21 2JN 
011-44-1962-818679 
LinkedIn https://www.linkedin.com/profile/view?id=60628729 
ResearchGate https://www.researchgate.net/profile/T_Ward2 

IBM System BlueGene Research 
IBM System BlueGene Marketing 

IBM Resources for Global Servants 
IBM Branded Products IBM Branded Swag 


UNIX in the Cloud - Find A Place Where There's Room To Grow, with the 
original Open Standard. Free Trial Here Today 
New Lamps For Old - Diskless Remote Boot Linux from National Center for 
High-Performance Computing, Taiwan 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Makefile.am
Description: Binary data
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] Reminder: assign as well as request review

[r...@open-mpi.org]

Hey folks

Just a reminder. If you request a review from someone, GitHub doesn’t show that 
person’s icon when looking at the list of PRs. It only shows their icon and 
marks the PR with their ID if you actually “assign” it to that person. Thus, 
just requesting a review without assigning the PR to someone makes it 
impossible for them to see which PRs are awaiting their attention.

Speaking personally, I have no idea which PRs are awaiting my attention unless 
you assign them to me. So please remember to do so.

Thanks
Ralph

___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

Re: [OMPI devel] CUDA kernels in OpenMPI

[Dmitry N. Mikushin]

It's hard to tell without complete makefile example. Could you please post
a minimal reprocase?

Note specifically for OpenMPI there is a tricky workaround. You can use
nvcc as mpicc compiler by exporting OMPI_CC=nvcc and wrapping out
incompatible compiler options.

Kind regards,
- Dmitry Mikushin.


2017-01-27 17:14 GMT+03:00 Chris Ward :

> I'm trying to build a CUDA kernel into OpenMPI (because I'm experimenting
> with an Allreduce collective with data in GPU buffers, and I want the GPU
> to do the reduction).
>
> This involves writing a '.cu' file, and compiling this to '.o' with the
> NVIDIA CUDA compiler 'nvcc'; and also writing some of OpenMPI in C++ to
> interface with the CUDA kernel.
>
> I've revised the Makefile.am in the relevant directory to look like (there
> is more, but this is the relevant section)
> sources = \
> coll_ibm.h \
> coll_ibm_component.c \
> coll_ibm_module.c \
> coll_ibm_allgather.c \
> coll_ibm_allgatherv.c \
> coll_ibm_allreduce.c \
> coll_ibm_alltoall.c \
> coll_ibm_alltoallv.c \
> coll_ibm_barrier.c \
> coll_ibm_bcast.c \
> coll_ibm_exscan.c \
> coll_ibm_gather.c \
> coll_ibm_gatherv.c \
> coll_ibm_reduce.c \
> coll_ibm_reduce_scatter.c \
> coll_ibm_reduce_scatter_block.c \
> coll_ibm_scan.c \
> coll_ibm_scatter.c \
> coll_ibm_scatterv.c \
> allreduce_overlap.cc \
> comm_gpu.cu
>
> SUFFIXES = .cu
>
> comm_gpu.lo: comm_gpu.cu
> /usr/local/cuda/bin/nvcc -gencode arch=compute_60,code=sm_60
> -lcuda -O3 --compiler-options "-O2 -fopenmp -mcpu=power8 -fPIC" -c
> comm_gpu.cu
> mv comm_gpu.o comm_gpu.lo
>
>
> The C++ compile runs OK; but the attempt to run 'nvcc' is ignored; the
> build completed, but no comm_gpu.lo is generated, and as far as I can tell,
> the library for this directory is built without the comm_gpu.lo object.
>
> What do I need to put in the Makefile.am to get this to build properly ?
>
>
>
>
> *T J (Chris) Ward, IBM Research. Scalable Data-Centric Computing - Active
> Storage Fabrics - IBM System BlueGene IBM United Kingdom Ltd., Hursley
> Park, Winchester, Hants, SO21 2JN 011-44-1962-818679 <+44%201962%20818679>
> LinkedIn **https://www.linkedin.com/profile/view?id=60628729*
> 
> * ResearchGate **https://www.researchgate.net/profile/T_Ward2*
> 
>
> *IBM System BlueGene Research* 
> *IBM System BlueGene Marketing*
> 
>
> *IBM Resources for Global Servants* 
> *IBM Branded Products*  *IBM Branded Swag*
> 
> 
>
> UNIX in the Cloud - Find A Place Where There's Room To Grow, with the
> original Open Standard. *Free Trial Here Today*
> 
> New Lamps For Old - *Diskless Remote Boot Linux*
>  from *National Center for High-Performance
> Computing, Taiwan* 
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
> ___
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
>
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel

[OMPI devel] CUDA kernels in OpenMPI

[Chris Ward]

I'm trying to build a CUDA kernel into OpenMPI (because I'm experimenting 
with an Allreduce collective with data in GPU buffers, and I want the GPU 
to do the reduction).

This involves writing a '.cu' file, and compiling this to '.o' with the 
NVIDIA CUDA compiler 'nvcc'; and also writing some of OpenMPI in C++ to 
interface with the CUDA kernel.

I've revised the Makefile.am in the relevant directory to look like (there 
is more, but this is the relevant section)
sources = \
coll_ibm.h \
coll_ibm_component.c \
coll_ibm_module.c \
coll_ibm_allgather.c \
coll_ibm_allgatherv.c \
coll_ibm_allreduce.c \
coll_ibm_alltoall.c \
coll_ibm_alltoallv.c \
coll_ibm_barrier.c \
coll_ibm_bcast.c \
coll_ibm_exscan.c \
coll_ibm_gather.c \
coll_ibm_gatherv.c \
coll_ibm_reduce.c \
coll_ibm_reduce_scatter.c \
coll_ibm_reduce_scatter_block.c \
coll_ibm_scan.c \
coll_ibm_scatter.c \
coll_ibm_scatterv.c \
allreduce_overlap.cc \
comm_gpu.cu

SUFFIXES = .cu

comm_gpu.lo: comm_gpu.cu
/usr/local/cuda/bin/nvcc -gencode arch=compute_60,code=sm_60 
-lcuda -O3 --compiler-options "-O2 -fopenmp -mcpu=power8 -fPIC" -c 
comm_gpu.cu
mv comm_gpu.o comm_gpu.lo


The C++ compile runs OK; but the attempt to run 'nvcc' is ignored; the 
build completed, but no comm_gpu.lo is generated, and as far as I can 
tell, the library for this directory is built without the comm_gpu.lo 
object.

What do I need to put in the Makefile.am to get this to build properly ?

T J (Chris) Ward, IBM Research. 
Scalable Data-Centric Computing - Active Storage Fabrics - IBM System 
BlueGene 
IBM United Kingdom Ltd., Hursley Park, Winchester, Hants, SO21 2JN 
011-44-1962-818679 
LinkedIn https://www.linkedin.com/profile/view?id=60628729 
ResearchGate https://www.researchgate.net/profile/T_Ward2 

IBM System BlueGene Research 
IBM System BlueGene Marketing 

IBM Resources for Global Servants 
IBM Branded Products IBM Branded Swag 


UNIX in the Cloud - Find A Place Where There's Room To Grow, with the 
original Open Standard. Free Trial Here Today 
New Lamps For Old - Diskless Remote Boot Linux from National Center for 
High-Performance Computing, Taiwan 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
___
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel