Re: [OMPI users] GPUDirect with OpenMPI

2015-03-09 Thread Aulwes, Rob
ith OpenMPI Let me try to reproduce this. This should not have anything to do with GPU Direct RDMA. However, to eliminate it, you could run with: --mca btl_openib_want_cuda_gdr 0. Rolf From: users [mailto:users-boun...@open-mpi.org] On Behalf Of Aulwes, Rob Sent: Wednesday, February 11, 2015 2:17 PM To:

[OMPI users] GPUDirect with OpenMPI

2015-02-11 Thread Aulwes, Rob
Hi, I built OpenMPI 1.8.3 using PGI 14.7 and enabled CUDA support for CUDA 6.0. I have a Fortran test code that tests GPUDirect and have included it here. When I run it across 2 nodes using 4 MPI procs, sometimes it fails with incorrect results. Specifically, sometimes rank 1 does not

[hwloc-users] binding to thread

2014-09-29 Thread Aulwes, Rob
Hi, I'm trying to allocate and bind memory on the same NUMA domain as the calling thread. The code I use is as follows. /* retrieve the single PU where the current thread actually runs within this process binding */ int err = hwloc_get_last_cpu_location(topo, set,

[hwloc-users] more detailed errors

2014-09-17 Thread Aulwes, Rob
Hi, A call to hwloc_topology_load is failing, but all that is returned is –1. Are there error reporting routines that can be called to get more details about the error? The doc for hwloc_topology_load only says that –1 is returned on error, but doesn't give any hints about what could cause

Re: [hwloc-users] setting memory bindings

2014-09-15 Thread Aulwes, Rob
to better show how to use bitmaps, cpu and memory binding etc. https://github.com/open-mpi/hwloc/tree/master/doc/examples If you see anything missing, don't hesitate to ask. Brice Le 19/08/2014 19:10, Aulwes, Rob a écrit : ok, in the meantime, is there a way to manually 'replicate'? Tha

Re: [hwloc-users] setting memory bindings

2014-09-02 Thread Aulwes, Rob
specific sets of cores) so you still need to check the return value :) I will better document all this, and what happens in case of failure with and without the STRICT flag. And I'll see if I add a good example somewhere. Brice Le 19/08/2014 19:00, Aulwes, Rob a écrit : nope, no error. is the

Re: [hwloc-users] setting memory bindings

2014-08-19 Thread Aulwes, Rob
oc(); /* iterate over remaining bits and alloc/bind there */ hwloc_bitmap_foreach_begin(i, totalset) { hwloc_bitmap_only(newset, i); p = hwloc_alloc_membind_nodeset(, newset, ...); } hwloc_bitmap_free(totalset); hwloc_bitmap_free(bindset); hwloc_bitmap_free(newset); Brice Le 19/08/

Re: [hwloc-users] setting memory bindings

2014-08-19 Thread Aulwes, Rob
ly. I'll look at clarifying the doc and/or returning an error and/or adding a way to find out. Brice Le 19/08/2014 19:00, Aulwes, Rob a écrit : nope, no error. is there a way to find out what policies are supported? I would like to try 'replicate'. From: Brice Goglin <brice.gog...@inria.

Re: [hwloc-users] setting memory bindings

2014-08-19 Thread Aulwes, Rob
t an error if you request it. Brice Le 19/08/2014 18:48, Aulwes, Rob a écrit : Hi Brice, I've tried: p = hwloc_alloc_membind_policy(_topo, cnt * sizeof (T), mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH, 0); and p = hwloc_alloc_me

[hwloc-users] setting memory bindings

2014-08-19 Thread Aulwes, Rob
Hi, I'm trying to write a custom C++ allocator that wraps hwloc calls. I've tried using various hwloc_alloc* functions to set the memory bindings, but when I call hwloc_get_area_membind_nodeset to verify, I don't get the same policy I passed to alloc. Are there example codes that show how to