I'll give this a try. Thanks Brice!
From: Brice Goglin <brice.gog...@inria.fr<mailto:brice.gog...@inria.fr>>
Reply-To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
List-Post: hwloc-users@lists.open-mpi.org
Date: Tue, 19 Aug 2014 19:26:17 +0200
To: <hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
Subject: Re: [hwloc-users] setting memory bindings
You have to pass HWLOC_MEMBIND_STRICT if you want an error code when the policy
isn't supported.
Assuming you get the nodeset of your current binding with
get_area_membind_nodeset() in bindset, you can do something like this
(untested):
hwloc_bitmap_t bindset, totalset, newset;
int i;
/* get the total nodeset and duplicate it before modifying it */
totalset = hwloc_bitmap_dup(hwloc_topology_get_topology_nodeset(topo));
/* get the current binding */
bindset = hwloc_bitmap_alloc();
hwloc_get_area_membind_nodeset(topology, ..., bindset, ...);
/* substract the nodeset where the buffer is already available */
hwloc_bitmap_andnot(totalset, totalset, bindset);
/* create a empty bitmap to play with inside the next loop */
newset = hwloc_bitmap_alloc();
/* iterate over remaining bits and alloc/bind there */
hwloc_bitmap_foreach_begin(i, totalset) {
hwloc_bitmap_only(newset, i);
p = hwloc_alloc_membind_nodeset(...., newset, ...);
}
hwloc_bitmap_free(totalset);
hwloc_bitmap_free(bindset);
hwloc_bitmap_free(newset);
Brice
Le 19/08/2014 19:10, Aulwes, Rob a écrit :
ok, in the meantime, is there a way to manually 'replicate'? That is, if I
allocate a node, I would like to find out which NUMA domain it resides in, and
then allocate replicates to other domains. Are there example codes that show
how to use the bitmaps for this? I've been unsuccessful in finding example
codes that illustrate the bitmaps. Did I miss it in any doc?
Thanks for the help! Rob
From: Brice Goglin <brice.gog...@inria.fr<mailto:brice.gog...@inria.fr>>
Reply-To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
List-Post: hwloc-users@lists.open-mpi.org
Date: Tue, 19 Aug 2014 19:03:56 +0200
To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
Subject: Re: [hwloc-users] setting memory bindings
REPLICATE isn't support either on Linux unfortunately.
I'll look at clarifying the doc and/or returning an error and/or adding a way
to find out.
Brice
Le 19/08/2014 19:00, Aulwes, Rob a écrit :
nope, no error. is there a way to find out what policies are supported? I
would like to try 'replicate'.
From: Brice Goglin <brice.gog...@inria.fr<mailto:brice.gog...@inria.fr>>
Reply-To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
List-Post: hwloc-users@lists.open-mpi.org
Date: Tue, 19 Aug 2014 18:55:57 +0200
To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
Subject: Re: [hwloc-users] setting memory bindings
NEXTTOUCH isn't supported on Linux. I hope you'd get an error if you request it.
Brice
Le 19/08/2014 18:48, Aulwes, Rob a écrit :
Hi Brice,
I've tried:
p = hwloc_alloc_membind_policy(_topo, cnt * sizeof (T),
mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH, 0);
and
p = hwloc_alloc_membind_nodeset(_topo, cnt * sizeof (T),
mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH, 0);
and
p = hwloc_alloc(_topo, cnt * sizeof (T));
hwloc_set_area_membind_nodeset(_topo, p, cnt * sizeof (T),
mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH,
0);
where
mem_nodeset = hwloc_topology_get_complete_nodeset(_topo);
Thanks,Rob
From: Brice Goglin <brice.gog...@inria.fr<mailto:brice.gog...@inria.fr>>
Reply-To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
List-Post: hwloc-users@lists.open-mpi.org
Date: Tue, 19 Aug 2014 18:44:05 +0200
To: Hardware locality user list
<hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>>
Subject: Re: [hwloc-users] setting memory bindings
Le 19/08/2014 18:38, Aulwes, Rob a écrit :
Hi,
I'm trying to write a custom C++ allocator that wraps hwloc calls. I've tried
using various hwloc_alloc* functions to set the memory bindings, but when I
call hwloc_get_area_membind_nodeset to verify, I don't get the same policy I
passed to alloc. Are there example codes that show how to set memory bindings?
More generally, are there example codes that help illustrate how to use hwloc
API?
Thanks,Rob Aulwes
Hello
Your problem may be related to the fact that same policies are identical on
some operating systems. For instance DEFAULT and FIRST_TOUCH are often
identical. What did you set and what did you get back?
There are many examples in the tests subdirectory, in doc/hwloc-helloc.c and in
hwloc/helper.h and hwloc/inline.h, but I don't think we have much about the
membind API, and this API isn't a easy one unfortunately (given the variety of
existing policies and operating systems).
Brice
_______________________________________________ hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org> Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1058.php
_______________________________________________
hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1058.php
_______________________________________________ hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org> Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1059.php
_______________________________________________
hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1060.php
_______________________________________________ hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org> Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1062.php
_______________________________________________
hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1063.php
_______________________________________________ hwloc-users mailing list
hwloc-us...@open-mpi.org<mailto:hwloc-us...@open-mpi.org> Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users Link to this post:
http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1063.php
