[hwloc-devel] Create success (hwloc r1.0a1r1306)

[MPI Team]

Creating nightly hwloc snapshot SVN tarball was a success.

Snapshot:   hwloc 1.0a1r1306
Start time: Thu Nov  5 21:01:05 EST 2009
End time:   Thu Nov  5 21:02:54 EST 2009

Your friendly daemon,
Cyrador

Re: [hwloc-devel] hwloc 0.9.2 autogen issue

[Pavan Balaji]

Thanks. That worked.

 -- Pavan

On 11/05/2009 01:56 PM, Jeff Squyres wrote:
> Ok, I see the issue.  Apply the attached patch and see if that works for
> you.
> 
> I'd prefer not to do another release today, if possible.  ;-)
> 
> 
> On Nov 5, 2009, at 2:50 PM, Jeff Squyres (jsquyres) wrote:
> 
>> (re-subjected to be accurate)
>>
>> I notice that my autogen.sh outputs:
>>
>> autoreconf: running: aclocal -I m4 --force -I config
>>
>> but yours outputs:
>>
>> autoreconf: running: aclocal --force -I config
>>
>>
>> where's the "-I m4"?
>>
>> ah, I see you're running AC 2.63.  I'm running AC 2.64.  I just
>> tried with AC 2.63 and I see the same problem.  Lemme see if I can get
>> a workaround.
>>
> 
> 
> 
> 
> 
> ___
> hwloc-devel mailing list
> hwloc-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] hwloc 0.9.2 autogen issue

[Jeff Squyres]

Ok, I see the issue.  Apply the attached patch and see if that works  
for you.


I'd prefer not to do another release today, if possible.  ;-)


On Nov 5, 2009, at 2:50 PM, Jeff Squyres (jsquyres) wrote:


(re-subjected to be accurate)

I notice that my autogen.sh outputs:

autoreconf: running: aclocal -I m4 --force -I config

but yours outputs:

autoreconf: running: aclocal --force -I config


where's the "-I m4"?

ah, I see you're running AC 2.63.  I'm running AC 2.64.  I just
tried with AC 2.63 and I see the same problem.  Lemme see if I can get
a workaround.




--
Jeff Squyres
jsquy...@cisco.com


ac263.patch
Description: Binary data

[hwloc-devel] hwloc 0.9.2 autogen issue

[Jeff Squyres]

(re-subjected to be accurate)

I notice that my autogen.sh outputs:

autoreconf: running: aclocal -I m4 --force -I config

but yours outputs:

autoreconf: running: aclocal --force -I config


where's the "-I m4"?

ah, I see you're running AC 2.63.  I'm running AC 2.64.  I just  
tried with AC 2.63 and I see the same problem.  Lemme see if I can get  
a workaround.



On Nov 5, 2009, at 2:18 PM, Pavan Balaji wrote:



Am I supposed to do something special here for hwloc-0.9.2?

% autoconf --version
autoconf (GNU Autoconf) 2.63
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

% libtool --version
ltmain.sh (GNU libtool) 2.2.6
Written by Gordon Matzigkeit , 1996

% ./autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I config
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
libtoolize: copying file `config/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `config'.
libtoolize: copying file `config/libtool.m4'
libtoolize: copying file `config/ltoptions.m4'
libtoolize: copying file `config/ltsugar.m4'
libtoolize: copying file `config/ltversion.m4'
libtoolize: copying file `config/lt~obsolete.m4'
autoreconf: running: /home/balaji/software/tools/install/bin/autoconf
--include=m4 --force
configure.ac:163: error: possibly undefined macro: AC_DEFINE
  If this token and others are legitimate, please use  
m4_pattern_allow.

  See the Autoconf documentation.
autoreconf: /home/balaji/software/tools/install/bin/autoconf failed  
with

exit status: 1

 -- Pavan

On 11/04/2009 03:55 PM, Jeff Squyres wrote:
> Please beat it up and look for problems!
>
> http://www.open-mpi.org/software/hwloc/v0.9/
>

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] 0.9.1rc4 is out

[Brice Goglin]

Pavan Balaji wrote:
> Sorry for going back-and-forth. I thought it was my error, but maybe it
> isn't. A fresh svn checkout from
> http://svn.open-mpi.org/svn/hwloc/tags/hwloc-v0.9.2 gives the error I
> mentioned.
>
> Am I missing something?
>   


What if you use the regular autoconf instead of your custom one ?

Brice

Re: [hwloc-devel] 0.9.1rc4 is out

[Pavan Balaji]

Sorry for going back-and-forth. I thought it was my error, but maybe it
isn't. A fresh svn checkout from
http://svn.open-mpi.org/svn/hwloc/tags/hwloc-v0.9.2 gives the error I
mentioned.

Am I missing something?

 -- Pavan

On 11/05/2009 01:22 PM, Pavan Balaji wrote:
> On 11/05/2009 01:21 PM, Jeff Squyres wrote:
>> FWIW, you shouldn't need to run autogen on the tarball (or are you
>> importing from svn?).
> 
> Importing from the svn.
> 
>  -- Pavan
> 

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] 0.9.1rc4 is out

[Pavan Balaji]

On 11/05/2009 01:21 PM, Jeff Squyres wrote:
> FWIW, you shouldn't need to run autogen on the tarball (or are you
> importing from svn?).

Importing from the svn.

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] 0.9.1rc4 is out

[Jeff Squyres]

FWIW, you shouldn't need to run autogen on the tarball (or are you  
importing from svn?).



On Nov 5, 2009, at 2:20 PM, Pavan Balaji wrote:



Apologies. This is my bad! My setup was messed up.

On 11/05/2009 01:18 PM, Pavan Balaji wrote:
> Am I supposed to do something special here for hwloc-0.9.2?
>
> % autoconf --version
> autoconf (GNU Autoconf) 2.63
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv2+: GNU GPL version 2 or later
> 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> % libtool --version
> ltmain.sh (GNU libtool) 2.2.6
> Written by Gordon Matzigkeit , 1996
>
> % ./autogen.sh
> autoreconf: Entering directory `.'
> autoreconf: configure.ac: not using Gettext
> autoreconf: running: aclocal --force -I config
> autoreconf: configure.ac: tracing
> autoreconf: running: libtoolize --copy --force
> libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
> libtoolize: copying file `config/ltmain.sh'
> libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `config'.
> libtoolize: copying file `config/libtool.m4'
> libtoolize: copying file `config/ltoptions.m4'
> libtoolize: copying file `config/ltsugar.m4'
> libtoolize: copying file `config/ltversion.m4'
> libtoolize: copying file `config/lt~obsolete.m4'
> autoreconf: running: /home/balaji/software/tools/install/bin/ 
autoconf

> --include=m4 --force
> configure.ac:163: error: possibly undefined macro: AC_DEFINE
>   If this token and others are legitimate, please use  
m4_pattern_allow.

>   See the Autoconf documentation.
> autoreconf: /home/balaji/software/tools/install/bin/autoconf  
failed with

> exit status: 1
>
>  -- Pavan
>
> On 11/04/2009 03:55 PM, Jeff Squyres wrote:
>> Please beat it up and look for problems!
>>
>> http://www.open-mpi.org/software/hwloc/v0.9/
>>
>

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] 0.9.1rc4 is out

[Pavan Balaji]

Apologies. This is my bad! My setup was messed up.

On 11/05/2009 01:18 PM, Pavan Balaji wrote:
> Am I supposed to do something special here for hwloc-0.9.2?
> 
> % autoconf --version
> autoconf (GNU Autoconf) 2.63
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv2+: GNU GPL version 2 or later
> 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> % libtool --version
> ltmain.sh (GNU libtool) 2.2.6
> Written by Gordon Matzigkeit , 1996
> 
> % ./autogen.sh
> autoreconf: Entering directory `.'
> autoreconf: configure.ac: not using Gettext
> autoreconf: running: aclocal --force -I config
> autoreconf: configure.ac: tracing
> autoreconf: running: libtoolize --copy --force
> libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
> libtoolize: copying file `config/ltmain.sh'
> libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `config'.
> libtoolize: copying file `config/libtool.m4'
> libtoolize: copying file `config/ltoptions.m4'
> libtoolize: copying file `config/ltsugar.m4'
> libtoolize: copying file `config/ltversion.m4'
> libtoolize: copying file `config/lt~obsolete.m4'
> autoreconf: running: /home/balaji/software/tools/install/bin/autoconf
> --include=m4 --force
> configure.ac:163: error: possibly undefined macro: AC_DEFINE
>   If this token and others are legitimate, please use m4_pattern_allow.
>   See the Autoconf documentation.
> autoreconf: /home/balaji/software/tools/install/bin/autoconf failed with
> exit status: 1
> 
>  -- Pavan
> 
> On 11/04/2009 03:55 PM, Jeff Squyres wrote:
>> Please beat it up and look for problems!
>>
>> http://www.open-mpi.org/software/hwloc/v0.9/
>>
> 

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] 0.9.1rc4 is out

[Pavan Balaji]

Am I supposed to do something special here for hwloc-0.9.2?

% autoconf --version
autoconf (GNU Autoconf) 2.63
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

% libtool --version
ltmain.sh (GNU libtool) 2.2.6
Written by Gordon Matzigkeit , 1996

% ./autogen.sh
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I config
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
libtoolize: copying file `config/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `config'.
libtoolize: copying file `config/libtool.m4'
libtoolize: copying file `config/ltoptions.m4'
libtoolize: copying file `config/ltsugar.m4'
libtoolize: copying file `config/ltversion.m4'
libtoolize: copying file `config/lt~obsolete.m4'
autoreconf: running: /home/balaji/software/tools/install/bin/autoconf
--include=m4 --force
configure.ac:163: error: possibly undefined macro: AC_DEFINE
  If this token and others are legitimate, please use m4_pattern_allow.
  See the Autoconf documentation.
autoreconf: /home/balaji/software/tools/install/bin/autoconf failed with
exit status: 1

 -- Pavan

On 11/04/2009 03:55 PM, Jeff Squyres wrote:
> Please beat it up and look for problems!
> 
> http://www.open-mpi.org/software/hwloc/v0.9/
> 

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

[hwloc-devel] towards PLPA-like API in 1.0

[Brice Goglin]

Hello,

I've been looking at the PLPA API and here's what we could do to add a
hwloc/plpa.h offering kind of the same features.



* PLPA-like API is prefixed with hwloc_plpa_ and all functions get a new
hwloc_topology_t parameter. The problematic ones are:

+ int hwloc_plpa_sched_getaffinity(pid_t pid, hwloc_cpuset_t cpuset);

It's just a hwloc_get_cpubind(), but we don't have it since it would not
be supported on all OS. But I think we should add it anyway.

+ int hwloc_plpa_get_core_flags(hwloc_topology_t topology, int socket_id, int 
core_id, int *exists, int *online);

Is says whether a core (given by core+socket os_index) exists and is
online. First, we don't have topology information about offline
processors. Secondly, on Nehalem you can disable a single thread within
a hyperthreaded core, so an "offline core" doesn't mean much. I would
just vote for returning whether the core exists and remove the online
return value here (see below for more about offline CPUs).

+ int hwloc_plpa_map_to_processor_id(hwloc_topology_t topology, int socket_id, 
int core_id, int *processor_id);

Return the os_index of the processor given by socket/core os_index. If
we have hyperthreading, all thread sibling are valid here. I would just
return the os_index of first one since this function was supposed to be
used for binding anyway.




* Then we have all count-spec related API, which lets you look for
information about all processors, or all online ones, or all offline ones.

If people are really interested with offline CPUs, they can look at the
get_offline_cpuset below. There is no topology information about offline
CPUs on Linux anyway, so I am not sure it's worth trying to manage
offline and online CPUs in a uniform way. I would rather remove the
count-spec argument and just only work on available/online/enabled
processors with:

+ int hwloc_plpa_get_processor_data(hwloc_topology_t topology, int 
*num_processors, int *max_processor_id);

+ int hwloc_plpa_get_processor_id(int processor_num, int *processor_id);




* Probing

>From what I understand, plpa_have_topology_information() tells whether
PLPA knows what's in the hardware, while plpa_api_probe() tells whether
binding is supported. We could add:

+ hwloc_topology_support(hwloc_topology_t topology, unsigned *support)

which fills "support" with a bitmask of things like OS is supported,
binding a thread is possible, binding a processor is possible, getting
the binding of a process is possible, ...

Then we could reimplement

+ int hwloc_plpa_have_topology_information(hwloc_topology_t topology);
+ int hwloc_plpa_api_probe(hwloc_topology_t topology);




* New hwloc general-purpose helpers that will be used by PLPA wrappers
but may still be useful for other reasons:

+ hwloc_obj_t hwloc_get_parent_obj_by_type(hwloc_topology_t topology, 
hwloc_obj_type_t type, hwloc_obj_t obj);
+ hwloc_obj_t hwloc_get_parent_obj_by_depth(hwloc_topology_t topology, unsigned 
depth, hwloc_obj_t obj);

Walks the parent chain until it finds the right type (or depth).

+ hwloc_obj_t hwloc_get_proc_obj_by_os_index(hwloc_topology_t topology, 
unsigned proc_os_index);

Given a processor os_index, returns the corresponding object.

+ hwloc_cpuset_t hwloc_topology_get_offline_cpuset(hwloc_topology_t topology);

Returns a CPU set of existing CPUs that are offline, disabled by
administrator, or unavailable to this process if we're restricting the
topology to the process origin binding for instance. I am not sure we
actually need to distinguish all these cases.




* Finally, I plan to reimplement the PLPA tools, either in tests/plpa/
or as a real (installed) tools for a transition period.

+ plpa-info already works in my tree. Are there people that really need
it? "lstopo -v -" basically shows the same and even more (offline CPUs
are not reported in the trunk but I modified my tree to print the number
of offline CPUs and the corresponding cpuset).

+ plpa-taskset needs a lot of work for convering its own cpuset stuff
into ours. It has an advanced binding syntax that some people may be
used to. hwloc-bind has an advanced but different syntax. Apart from
that, the features are the same.



By the way, I wonder if we want to add public functions converting
between cpusets (0x0f00) and cpulist string (0-15,24-27)
(plpa-taskset uses something like this).


Brice

Re: [hwloc-devel] Pgcc issues fixed?

[Jeff Squyres]

This problem may go away if we adapt PLPA's approach to sched_[set| 
get]affinity.



On Nov 4, 2009, at 10:34 PM, Chris Samuel wrote:



- "Chris Samuel"  wrote:

> - "Jeff Squyres"  wrote:
>
> > K.  Clear for a final rc / release?
>
> Go for it, am just about to go run a training course
> now so won't be available until this arvo Melbourne
> time..

Seems fine with PGI, Intel and GCC on AMD64, so I
thought I'd give it a whirl on our old SLES9 PPC64
cluster with XLC, that whinges about the usual params
unused, but also says:

"topology-linux.c", line 146.33: 1506-280 (W) Function argument  
assignment between types "unsigned int" and "struct {...}*" is not  
allowed.


cheers,
Chris
--
Christopher Samuel - (03) 9925 4751 - Systems Manager
 The Victorian Partnership for Advanced Computing
 P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency
___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1286

[Jeff Squyres]

I think we need another 2 weeks of testing!

:p


On Nov 5, 2009, at 3:31 AM, Brice Goglin wrote:


Of course, this is a very minor problem that doesn't need a rc5 and
doesn't prevent from releasing today. It's been a month and a half
already, we really need this first hwloc release out of door now (I
don't want to hear about Friday again :))

Brice



bgog...@osl.iu.edu wrote:
> Author: bgoglin
> Date: 2009-11-05 02:39:00 EST (Thu, 05 Nov 2009)
> New Revision: 1286
> URL: https://svn.open-mpi.org/trac/hwloc/changeset/1286
>
> Log:
> Backport trunk r1285:
>   Fix hwloc_cpuset_to_linux_libnuma_ulongs, thanks tests/linux- 
libnuma.c

>

___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1286

[Brice Goglin]

Of course, this is a very minor problem that doesn't need a rc5 and
doesn't prevent from releasing today. It's been a month and a half
already, we really need this first hwloc release out of door now (I
don't want to hear about Friday again :))

Brice



bgog...@osl.iu.edu wrote:
> Author: bgoglin
> Date: 2009-11-05 02:39:00 EST (Thu, 05 Nov 2009)
> New Revision: 1286
> URL: https://svn.open-mpi.org/trac/hwloc/changeset/1286
>
> Log:
> Backport trunk r1285:
>   Fix hwloc_cpuset_to_linux_libnuma_ulongs, thanks tests/linux-libnuma.c
>