[hwloc-devel] Query on the output of hwloc API

[Swati Agrawal]

Hi All,

I have recently started using hwloc and stuck with a case where there are
no NUMA nodes. I see that when i run "lscpu" command, it shows me there is
1 Numa Node and all the PUs are in this node.
But when i try reading the nodeset for my object using
hwloc_get_non_io_ancestor_obj(..), I see below output:

obj->nodeset->ulongs_count = 1;
obj->nodeset->ulongs[0] = 18446744073709551615 (UINT64 MAX Value).

What does this actually mean?

Thanks,
Swati
___
hwloc-devel mailing list
hwloc-devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel

Re: [hwloc-devel] Query on the output of hwloc API

[Brice Goglin]

Hello

If there's no NUMA node object in your hwloc topology, it means your
machine isn't NUMA (there's a single NUMA node), or your system doesn't
report NUMA information at all (missing NUMA support in the kernel, etc).

This is an old design choice that is not convenient. So we'll change
that in the upcoming hwloc 2.0. There will always be at least one NUMA
node object (just like in lspcu).

In the meantime, the meaning of obj->nodeset isn't very useful when
there's no NUMA object anyway. If you really need to look at
obj->nodeset on non-NUMA machines, you'll get either NULL or a "full"
"infinite" bitmap (meaning "the entire machine memory", as explained in
the description of the nodeset attribute of the object structure
https://www.open-mpi.org/projects/hwloc/doc/v1.11.4/a00038.php#a08f0d0e16c619a6e653526cbee4ffea3).

By the way, you're not supposed to look at internal nodeset fields
(ulongs and ulongs_count). For instance, there's another field saying
that the bitmap is infinite. All these are private details not meant to
be understood by users. Things like hwloc_bitmap_asprintf() or "lstopo
-.xml" would show that obj->nodeset is 0xf...f which means "infinite" or
"full".

Again, these infinite nodesets will go away in the upcoming hwloc 2.0.

Brice






Le 27/09/2016 01:35, Swati Agrawal a écrit :
> Hi All,
>
> I have recently started using hwloc and stuck with a case where there
> are no NUMA nodes. I see that when i run "lscpu" command, it shows me
> there is 1 Numa Node and all the PUs are in this node.
> But when i try reading the nodeset for my object using
> hwloc_get_non_io_ancestor_obj(..), I see below output:
>
> obj->nodeset->ulongs_count = 1;
> obj->nodeset->ulongs[0] = 18446744073709551615 (UINT64 MAX Value).
>
> What does this actually mean?
>
> Thanks,
> Swati
>
>
> ___
> hwloc-devel mailing list
> hwloc-devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel

___
hwloc-devel mailing list
hwloc-devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel

Re: [hwloc-devel] Query on the output of hwloc API

[Swati Agrawal]

Thanks Brice for the detailed info.
So, can I say that if obj->nodeset is all 1s, it is a no NUMA mode setup?

Thanks,
Swati

On Monday, September 26, 2016, Brice Goglin  wrote:

> Hello
>
> If there's no NUMA node object in your hwloc topology, it means your
> machine isn't NUMA (there's a single NUMA node), or your system doesn't
> report NUMA information at all (missing NUMA support in the kernel, etc).
>
> This is an old design choice that is not convenient. So we'll change that
> in the upcoming hwloc 2.0. There will always be at least one NUMA node
> object (just like in lspcu).
>
> In the meantime, the meaning of obj->nodeset isn't very useful when
> there's no NUMA object anyway. If you really need to look at obj->nodeset
> on non-NUMA machines, you'll get either NULL or a "full" "infinite" bitmap
> (meaning "the entire machine memory", as explained in the description of
> the nodeset attribute of the object structure https://www.open-mpi.org/
> projects/hwloc/doc/v1.11.4/a00038.php#a08f0d0e16c619a6e653526cbee4ffea3).
>
> By the way, you're not supposed to look at internal nodeset fields (ulongs
> and ulongs_count). For instance, there's another field saying that the
> bitmap is infinite. All these are private details not meant to be
> understood by users. Things like hwloc_bitmap_asprintf() or "lstopo -.xml"
> would show that obj->nodeset is 0xf...f which means "infinite" or "full".
>
> Again, these infinite nodesets will go away in the upcoming hwloc 2.0.
>
> Brice
>
>
>
>
>
>
> Le 27/09/2016 01:35, Swati Agrawal a écrit :
>
> Hi All,
>
> I have recently started using hwloc and stuck with a case where there are
> no NUMA nodes. I see that when i run "lscpu" command, it shows me there is
> 1 Numa Node and all the PUs are in this node.
> But when i try reading the nodeset for my object using
> hwloc_get_non_io_ancestor_obj(..), I see below output:
>
> obj->nodeset->ulongs_count = 1;
> obj->nodeset->ulongs[0] = 18446744073709551615 (UINT64 MAX Value).
>
> What does this actually mean?
>
> Thanks,
> Swati
>
>
> ___
> hwloc-devel mailing listhwloc-de...@lists.open-mpi.org 
> https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel
>
>
>
___
hwloc-devel mailing list
hwloc-devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel

Re: [hwloc-devel] Query on the output of hwloc API

[Brice Goglin]

To be future-proof (so that your code works both with current hwloc 1.x
and upcoming 2.0), the best check for non-NUMA machines is
   hwloc_get_nbobjs_by_type(topology, HWLOC_OBJ_NODE) <= 1

Otherwise, yes. Infinite/full nodeset means no NUMA. To actually check
whether a nodeset is infinite, use hwloc_bitmap_weight(obj->nodeset). It
returns -1 on infinite/full bitmaps since there's no way to count an
infinite set of bits.

Brice


Le 27/09/2016 07:45, Swati Agrawal a écrit :
> Thanks Brice for the detailed info.
> So, can I say that if obj->nodeset is all 1s, it is a no NUMA mode setup?
>
> Thanks,
> Swati
>
> On Monday, September 26, 2016, Brice Goglin  > wrote:
>
> Hello
>
> If there's no NUMA node object in your hwloc topology, it means
> your machine isn't NUMA (there's a single NUMA node), or your
> system doesn't report NUMA information at all (missing NUMA
> support in the kernel, etc).
>
> This is an old design choice that is not convenient. So we'll
> change that in the upcoming hwloc 2.0. There will always be at
> least one NUMA node object (just like in lspcu).
>
> In the meantime, the meaning of obj->nodeset isn't very useful
> when there's no NUMA object anyway. If you really need to look at
> obj->nodeset on non-NUMA machines, you'll get either NULL or a
> "full" "infinite" bitmap (meaning "the entire machine memory", as
> explained in the description of the nodeset attribute of the
> object structure
> 
> https://www.open-mpi.org/projects/hwloc/doc/v1.11.4/a00038.php#a08f0d0e16c619a6e653526cbee4ffea3
> 
> ).
>
> By the way, you're not supposed to look at internal nodeset fields
> (ulongs and ulongs_count). For instance, there's another field
> saying that the bitmap is infinite. All these are private details
> not meant to be understood by users. Things like
> hwloc_bitmap_asprintf() or "lstopo -.xml" would show that
> obj->nodeset is 0xf...f which means "infinite" or "full".
>
> Again, these infinite nodesets will go away in the upcoming hwloc 2.0.
>
> Brice
>
>
>
>
>
>
> Le 27/09/2016 01:35, Swati Agrawal a écrit :
>> Hi All,
>>
>> I have recently started using hwloc and stuck with a case where
>> there are no NUMA nodes. I see that when i run "lscpu" command,
>> it shows me there is 1 Numa Node and all the PUs are in this node.
>> But when i try reading the nodeset for my object using
>> hwloc_get_non_io_ancestor_obj(..), I see below output:
>>
>> obj->nodeset->ulongs_count = 1;
>> obj->nodeset->ulongs[0] = 18446744073709551615 (UINT64 MAX Value).
>>
>> What does this actually mean?
>>
>> Thanks,
>> Swati
>>
>>
>> ___
>> hwloc-devel mailing list
>> hwloc-devel@lists.open-mpi.org
>> 
>> https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel
>> 
>
> ___
> hwloc-devel mailing list
> hwloc-devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel
___
hwloc-devel mailing list
hwloc-devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel

Re: [hwloc-devel] Query on the output of hwloc API

[Swati Agrawal]

Sure!. Great thanks Brice.


On Mon, Sep 26, 2016 at 10:58 PM, Brice Goglin 
wrote:

> To be future-proof (so that your code works both with current hwloc 1.x
> and upcoming 2.0), the best check for non-NUMA machines is
>hwloc_get_nbobjs_by_type(topology, HWLOC_OBJ_NODE) <= 1
>
> Otherwise, yes. Infinite/full nodeset means no NUMA. To actually check
> whether a nodeset is infinite, use hwloc_bitmap_weight(obj->nodeset). It
> returns -1 on infinite/full bitmaps since there's no way to count an
> infinite set of bits.
>
> Brice
>
>
>
> Le 27/09/2016 07:45, Swati Agrawal a écrit :
>
> Thanks Brice for the detailed info.
> So, can I say that if obj->nodeset is all 1s, it is a no NUMA mode setup?
>
> Thanks,
> Swati
>
> On Monday, September 26, 2016, Brice Goglin < 
> brice.gog...@inria.fr> wrote:
>
>> Hello
>>
>> If there's no NUMA node object in your hwloc topology, it means your
>> machine isn't NUMA (there's a single NUMA node), or your system doesn't
>> report NUMA information at all (missing NUMA support in the kernel, etc).
>>
>> This is an old design choice that is not convenient. So we'll change that
>> in the upcoming hwloc 2.0. There will always be at least one NUMA node
>> object (just like in lspcu).
>>
>> In the meantime, the meaning of obj->nodeset isn't very useful when
>> there's no NUMA object anyway. If you really need to look at obj->nodeset
>> on non-NUMA machines, you'll get either NULL or a "full" "infinite" bitmap
>> (meaning "the entire machine memory", as explained in the description of
>> the nodeset attribute of the object structure
>> https://www.open-mpi.org/projects/hwloc/doc/v1.11.4/a00038.
>> php#a08f0d0e16c619a6e653526cbee4ffea3).
>>
>> By the way, you're not supposed to look at internal nodeset fields
>> (ulongs and ulongs_count). For instance, there's another field saying that
>> the bitmap is infinite. All these are private details not meant to be
>> understood by users. Things like hwloc_bitmap_asprintf() or "lstopo -.xml"
>> would show that obj->nodeset is 0xf...f which means "infinite" or "full".
>>
>> Again, these infinite nodesets will go away in the upcoming hwloc 2.0.
>>
>> Brice
>>
>>
>>
>>
>>
>>
>> Le 27/09/2016 01:35, Swati Agrawal a écrit :
>>
>> Hi All,
>>
>> I have recently started using hwloc and stuck with a case where there are
>> no NUMA nodes. I see that when i run "lscpu" command, it shows me there is
>> 1 Numa Node and all the PUs are in this node.
>> But when i try reading the nodeset for my object using
>> hwloc_get_non_io_ancestor_obj(..), I see below output:
>>
>> obj->nodeset->ulongs_count = 1;
>> obj->nodeset->ulongs[0] = 18446744073709551615 (UINT64 MAX Value).
>>
>> What does this actually mean?
>>
>> Thanks,
>> Swati
>>
>>
>> ___
>> hwloc-devel mailing 
>> listhwloc-de...@lists.open-mpi.orghttps://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel
>>
>> ___
> hwloc-devel mailing 
> listhwloc-de...@lists.open-mpi.orghttps://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel
>
>
> ___
> hwloc-devel mailing list
> hwloc-devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel
>
___
hwloc-devel mailing list
hwloc-devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/hwloc-devel