You have to pass HWLOC_MEMBIND_STRICT if you want an error code when the
policy isn't supported.
Assuming you get the nodeset of your current binding with
get_area_membind_nodeset() in bindset, you can do something like this
(untested):
hwloc_bitmap_t bindset, totalset, newset;
int i;
/* get the total nodeset and duplicate it before modifying it */
totalset = hwloc_bitmap_dup(hwloc_topology_get_topology_nodeset(topo));
/* get the current binding */
bindset = hwloc_bitmap_alloc();
hwloc_get_area_membind_nodeset(topology, ..., bindset, ...);
/* substract the nodeset where the buffer is already available */
hwloc_bitmap_andnot(totalset, totalset, bindset);
/* create a empty bitmap to play with inside the next loop */
newset = hwloc_bitmap_alloc();
/* iterate over remaining bits and alloc/bind there */
hwloc_bitmap_foreach_begin(i, totalset) {
hwloc_bitmap_only(newset, i);
p = hwloc_alloc_membind_nodeset(...., newset, ...);
}
hwloc_bitmap_free(totalset);
hwloc_bitmap_free(bindset);
hwloc_bitmap_free(newset);
Brice
Le 19/08/2014 19:10, Aulwes, Rob a écrit :
> ok, in the meantime, is there a way to manually 'replicate'? That is,
> if I allocate a node, I would like to find out which NUMA domain it
> resides in, and then allocate replicates to other domains. Are there
> example codes that show how to use the bitmaps for this? I've been
> unsuccessful in finding example codes that illustrate the bitmaps.
> Did I miss it in any doc?
>
> Thanks for the help! Rob
>
> From: Brice Goglin <brice.gog...@inria.fr <mailto:brice.gog...@inria.fr>>
> Reply-To: Hardware locality user list <hwloc-us...@open-mpi.org
> <mailto:hwloc-us...@open-mpi.org>>
> Date: Tue, 19 Aug 2014 19:03:56 +0200
> To: Hardware locality user list <hwloc-us...@open-mpi.org
> <mailto:hwloc-us...@open-mpi.org>>
> Subject: Re: [hwloc-users] setting memory bindings
>
> REPLICATE isn't support either on Linux unfortunately.
>
> I'll look at clarifying the doc and/or returning an error and/or
> adding a way to find out.
>
> Brice
>
>
>
> Le 19/08/2014 19:00, Aulwes, Rob a écrit :
>> nope, no error. is there a way to find out what policies are
>> supported? I would like to try 'replicate'.
>>
>> From: Brice Goglin <brice.gog...@inria.fr <mailto:brice.gog...@inria.fr>>
>> Reply-To: Hardware locality user list <hwloc-us...@open-mpi.org
>> <mailto:hwloc-us...@open-mpi.org>>
>> Date: Tue, 19 Aug 2014 18:55:57 +0200
>> To: Hardware locality user list <hwloc-us...@open-mpi.org
>> <mailto:hwloc-us...@open-mpi.org>>
>> Subject: Re: [hwloc-users] setting memory bindings
>>
>> NEXTTOUCH isn't supported on Linux. I hope you'd get an error if you
>> request it.
>>
>> Brice
>>
>>
>>
>> Le 19/08/2014 18:48, Aulwes, Rob a écrit :
>>> Hi Brice,
>>>
>>> I've tried:
>>>
>>> p = hwloc_alloc_membind_policy(_topo, cnt * sizeof
>>> (T),
>>> mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH, 0);
>>>
>>> and
>>>
>>> p = hwloc_alloc_membind_nodeset(_topo, cnt *
>>> sizeof (T),
>>> mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH, 0);
>>>
>>> and
>>>
>>>
>>> p = hwloc_alloc(_topo, cnt * sizeof (T));
>>> hwloc_set_area_membind_nodeset(_topo, p, cnt * sizeof (T),
>>>
>>> mem_nodeset, HWLOC_MEMBIND_NEXTTOUCH, 0);
>>>
>>> where
>>>
>>> mem_nodeset = hwloc_topology_get_complete_nodeset(_topo);
>>>
>>> Thanks,Rob
>>>
>>> From: Brice Goglin <brice.gog...@inria.fr
>>> <mailto:brice.gog...@inria.fr>>
>>> Reply-To: Hardware locality user list <hwloc-us...@open-mpi.org
>>> <mailto:hwloc-us...@open-mpi.org>>
>>> Date: Tue, 19 Aug 2014 18:44:05 +0200
>>> To: Hardware locality user list <hwloc-us...@open-mpi.org
>>> <mailto:hwloc-us...@open-mpi.org>>
>>> Subject: Re: [hwloc-users] setting memory bindings
>>>
>>> Le 19/08/2014 18:38, Aulwes, Rob a écrit :
>>>> Hi,
>>>>
>>>> I'm trying to write a custom C++ allocator that wraps hwloc calls.
>>>> I've tried using various hwloc_alloc* functions to set the memory
>>>> bindings, but when I call hwloc_get_area_membind_nodeset to verify,
>>>> I don't get the same policy I passed to alloc. Are there example
>>>> codes that show how to set memory bindings? More generally, are
>>>> there example codes that help illustrate how to use hwloc API?
>>>>
>>>> Thanks,Rob Aulwes
>>>>
>>>
>>> Hello
>>>
>>> Your problem may be related to the fact that same policies are
>>> identical on some operating systems. For instance DEFAULT and
>>> FIRST_TOUCH are often identical. What did you set and what did you
>>> get back?
>>>
>>> There are many examples in the tests subdirectory, in
>>> doc/hwloc-helloc.c and in hwloc/helper.h and hwloc/inline.h, but I
>>> don't think we have much about the membind API, and this API isn't a
>>> easy one unfortunately (given the variety of existing policies and
>>> operating systems).
>>>
>>> Brice
>>>
>>> _______________________________________________ hwloc-users mailing
>>> list hwloc-us...@open-mpi.org <mailto:hwloc-us...@open-mpi.org>
>>> Subscription:
>>> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users Link to
>>> this post:
>>> http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1058.php
>>>
>>>
>>> _______________________________________________
>>> hwloc-users mailing list
>>> hwloc-us...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1058.php
>>
>> _______________________________________________ hwloc-users mailing
>> list hwloc-us...@open-mpi.org <mailto:hwloc-us...@open-mpi.org>
>> Subscription:
>> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users Link to this
>> post:
>> http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1059.php
>>
>>
>> _______________________________________________
>> hwloc-users mailing list
>> hwloc-us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>> Link to this post:
>> http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1060.php
>
> _______________________________________________ hwloc-users mailing
> list hwloc-us...@open-mpi.org <mailto:hwloc-us...@open-mpi.org>
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> <http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users> Link to
> this post:
> http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1062.php
>
>
> _______________________________________________
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> Link to this post:
> http://www.open-mpi.org/community/lists/hwloc-users/2014/08/1063.php
