Re: [hwloc-devel] python-hwloc available via pip install
Yes, gitlab currently has bug rendering restructuredtext code blocks. You can click the "raw" button, or see https://gitlab.com/guystreeter/python-hwloc/raw/master/README.rst thanks, --Guy On Wed, Jan 3, 2018 at 4:15 PM Jirka Hladky wrote: > Hi Guy, > > please note that the examples are not visible in the rendered file (the > default view). Please check the attached snapshot. > > Thanks a lot! > Jirka > > On Wed, Jan 3, 2018 at 11:03 PM, Guy Streeter > wrote: > >> I've made python-hwloc available to install through pip. It's a source >> distribution, so development packages have to be installed first. I have >> given instructions for Fedora 26 an 27 and Debian9 in the README.rst file >> in the source tree. >> See >> https://gitlab.com/guystreeter/python-hwloc/blob/master/README.rst >> >> --Guy >> >> >> ___ >> hwloc-devel mailing list >> hwloc-devel@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/hwloc-devel >> > > ___ > hwloc-devel mailing list > hwloc-devel@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-devel ___ hwloc-devel mailing list hwloc-devel@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-devel
[hwloc-devel] python-hwloc available via pip install
I've made python-hwloc available to install through pip. It's a source distribution, so development packages have to be installed first. I have given instructions for Fedora 26 an 27 and Debian9 in the README.rst file in the source tree. See https://gitlab.com/guystreeter/python-hwloc/blob/master/README.rst --Guy ___ hwloc-devel mailing list hwloc-devel@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-devel
Re: [hwloc-devel] Python bindings for hwloc
I looked at implementing the bindings for the CUDA driver library interfaces, and I can't see a way to add them without implementing (at least partially) Python bindings for the CUDA driver library. That's too much work for a retired person to take on. :) The CUDA runtime library seems a bit easier. I've created a git branch called "cudart" which I hope has a proper implementation of those bindings. I can't build it because hwloc's cudart.h needs cuda.h from the CUDA driver development files, which I do not have. On an RPM based system you should be able to "make rpm". On other platforms the setup.py should build the bindings for you to try. I implemented test/cudart.py, patterned after the test/cudart.c from hwloc. --Guy On Sat, Sep 2, 2017 at 6:42 AM Jeff Squyres (jsquyres) wrote: > Guy -- > > This is tremendous. Thank you! > > > > On Sep 1, 2017, at 6:32 PM, Guy Streeter wrote: > > > > I'm still retired, but I found time to update python-hwloc for hwloc > version 1.11.5, the version currently shipped in Fedora 26. I made some > bug-fixes along the way, and built new versions for Fedora 25 and Cento 7 > as well. > > > > There are 2 important hosting changes for python-hwloc: the fdeorahosted > git server has been deactivated, and I no longer host files at redhat.com. > > > > The new location of the python-hwloc git tree is > > https://gitlab.com/guystreeter/python-hwloc > > > > RPM repos for Fedora 25 and 26 and Centos 7 (EPEL) are found at > > https://copr.fedorainfracloud.org/coprs/streeter/python-hwloc/ > > > > Other important changes: > > > > Documentation! I wrote a programmer's guide of sorts. It describes all > the classes and their methods, and has a few examples. > > > > For non-RPM distros, setup.py should now be usable to build and install > python-hwloc. Just run it with python2 or python3 (or both) to build the > correct version. Building requires Cython, and the development files for > several other packages. See the lines containing "BuildRequires" in the > top-level python-hwloc.spec file for a list. > > > > One last note about the state of python-hwloc development: I don't have > access to a system with CUDA development files or CUDA devices, so I have > not completed the implementation of CUDA-related features. > > > > I'm in the process of writing (slowly, I'm still retired) a GUI program > in Python to assist in setting process affinities. I'll post when I have it > in a usable state. > > > > Let me know if you are interested in seeing any changes, or have any > problems, in python-hwloc. > > > > regards, > > --Guy Streeter > > > > ___ > > hwloc-devel mailing list > > hwloc-devel@lists.open-mpi.org > > https://lists.open-mpi.org/mailman/listinfo/hwloc-devel > > > -- > Jeff Squyres > jsquy...@cisco.com > > ___ > hwloc-devel mailing list > hwloc-devel@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/hwloc-devel > ___ hwloc-devel mailing list hwloc-devel@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-devel
[hwloc-devel] Python bindings for hwloc
I'm still retired, but I found time to update python-hwloc for hwloc version 1.11.5, the version currently shipped in Fedora 26. I made some bug-fixes along the way, and built new versions for Fedora 25 and Cento 7 as well. There are 2 important hosting changes for python-hwloc: the fdeorahosted git server has been deactivated, and I no longer host files at redhat.com. The new location of the python-hwloc git tree is https://gitlab.com/guystreeter/python-hwloc RPM repos for Fedora 25 and 26 and Centos 7 (EPEL) are found at https://copr.fedorainfracloud.org/coprs/streeter/python-hwloc/ Other important changes: Documentation! I wrote a programmer's guide of sorts. It describes all the classes and their methods, and has a few examples. For non-RPM distros, setup.py should now be usable to build and install python-hwloc. Just run it with python2 or python3 (or both) to build the correct version. Building requires Cython, and the development files for several other packages. See the lines containing "BuildRequires" in the top-level python-hwloc.spec file for a list. One last note about the state of python-hwloc development: I don't have access to a system with CUDA development files or CUDA devices, so I have not completed the implementation of CUDA-related features. I'm in the process of writing (slowly, I'm still retired) a GUI program in Python to assist in setting process affinities. I'll post when I have it in a usable state. Let me know if you are interested in seeing any changes, or have any problems, in python-hwloc. regards, --Guy Streeter ___ hwloc-devel mailing list hwloc-devel@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/hwloc-devel
Re: [hwloc-devel] Check the return value of getenv() in, hwloc_plugin_check_namespace
On 10/09/2015 04:31 PM, Brice Goglin wrote: > This broken code is only used when a plugin fails to find core symbols. > Only happens in case of namespace issues (for instance when libhwloc is > loaded by another layer of plugin, something we advice not to do). I don't think I'm doing that, but I'm not sure exactly what that means. It fails in python-hwloc consistently on a Fedora 22 virtual machine, but not at all on Fedora 21 on real hardware. (Those are my 2 test-cases) --Guy > > Thanks, I'll apply this. > > Brice > > > > Le 09/10/2015 23:04, Guy Streeter a écrit : >> I am not able explain why this doesn't fail everywhere. If >> HWLOC_PLUGINS_VERBOSE is not set, atoi() gets called with a NULL pointer, and >> the behavior in that case is undocumented. >> >> --Guy >> >> >> ___ >> hwloc-devel mailing list >> hwloc-de...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel >> Link to this post: >> http://www.open-mpi.org/community/lists/hwloc-devel/2015/10/4627.php > > > > > ___ > hwloc-devel mailing list > hwloc-de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel > Link to this post: > http://www.open-mpi.org/community/lists/hwloc-devel/2015/10/4629.php >
[hwloc-devel] Check the return value of getenv() in, hwloc_plugin_check_namespace
I am not able explain why this doesn't fail everywhere. If
HWLOC_PLUGINS_VERBOSE is not set, atoi() gets called with a NULL pointer, and
the behavior in that case is undocumented.
--Guy
>From 36a79abe46356ff360e192039fae3829b34b7997 Mon Sep 17 00:00:00 2001
From: Guy Streeter
List-Post: hwloc-devel@lists.open-mpi.org
Date: Fri, 9 Oct 2015 15:49:15 -0500
Subject: [PATCH] Check the return value of getenv() in
hwloc_plugin_check_namespace
---
include/hwloc/plugins.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/hwloc/plugins.h b/include/hwloc/plugins.h
index dd4e46a..ef78806 100644
--- a/include/hwloc/plugins.h
+++ b/include/hwloc/plugins.h
@@ -353,7 +353,7 @@ hwloc_plugin_check_namespace(const char *pluginname __hwloc_attribute_unused, co
static int verboseenv_value = 0;
if (!verboseenv_checked) {
const char *verboseenv = getenv("HWLOC_PLUGINS_VERBOSE");
- verboseenv_value = atoi(verboseenv);
+ verboseenv_value = verboseenv ? atoi(verboseenv) : 0;
verboseenv_checked = 1;
}
if (verboseenv_value)
--
2.1.0
Re: [hwloc-devel] Using hwloc to detect Hard Disks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/22/2014 05:55 PM, Brice Goglin wrote: > Le 22/09/2014 23:28, Ralph Castain a écrit : >> >> I believe that is what we are requesting - and to be clear, Intel is >> proposing to develop and contribute this feature. We'd like to add >> attributes to the objects to record: >> >> * memory - the serial number and model number of the DIMMs >> >> * disks - serial, model, manufacturer and any other available info >> (sometimes they have spindle speed, for example) >> >> HWLOC already provides similar info for processors and mother boards, >> so it seemed a natural extension of current capabilities to provide it >> for other system elements. > > Disk vendor/model is easy to add from sysfs on Linux. I don't know where > to find the serial number. Spindle speed may require more than just > sysfs. Do you have more info on how to get these attributes? > I know that udev gathers this information: # ll /sys/block/sda/bdi lrwxrwxrwx. 1 root root 0 Sep 23 09:33 /sys/block/sda/bdi -> ../../../../../../../../virtual/bdi/8:0 # grep SERIAL '/run/udev/data/b8:0' E:ID_SERIAL=SAMSUNG_MZ7TD256HAFV-000L9_S17LNSADC13325 E:ID_SERIAL_SHORT=S17LNSADC13325 So you could get it from udev or gather it the same way udev does. If you want to know how udev does it, I can research that. - --Guy -BEGIN PGP SIGNATURE- Version: GnuPG v1 iEYEARECAAYFAlQhhfsACgkQ0Bme0QyNhPSlNwCgt7Io0T60goabPNF5zEiOxgxM nckAnjsm2JMJOEP5p5/Sk9wyx/f4G9zX =2iEW -END PGP SIGNATURE-
[hwloc-devel] cython re-write of python-hwloc
Partly to prepare for the eventual switch to python3, and partly for the better refcount handling etc., I have rewritten python-hwloc and the requisite python-libnuma in Cython. The only drawback I've noticed with this change is that where python-hwloc used to be backward-compatible with older hwloc releases, it now is bound to the particular hwloc library version it was built against. source tarballs and Fedora RPMs are at http://people.redhat.com/streeter/ and the git repos are https://git.fedorahosted.org/cgit/python-libnuma.git/ and https://git.fedorahosted.org/cgit/python-hwloc.git/ --Guy
Re: [hwloc-devel] python bindings testing help needed
On 06/05/2013 12:29 AM, Brice Goglin wrote: > Le 05/06/2013 00:03, Guy Streeter a écrit : ... >> I think my regular expression code is broken in the gl stuff. Do you have to >> have the hardware and drivers for GL, or could I use an XML file for testing? >> I believe I have the regex for GL fixed now. I've updated my git (1.7-devel branch), and put a new tarfile on my people page git://git.fedorahosted.org/python-hwloc.git http://people.redhat.com/streeter/ --Guy
Re: [hwloc-devel] python bindings testing help needed
On 06/04/2013 03:52 AM, Brice Goglin wrote: > (forgot to CC the list) > > > Le 04/06/2013 10:48, Brice Goglin a écrit : >> Hello, >> >> Here are some slightly fixed tests. cuda/cudart/nvml look good. >> intel-mic looks good but I couldn't test on a real machine (I used XML >> instead), so the cpuset retrieving code wasn't tested. >> >> gl doesn't seem to find any GPU, but I don't know enough about Python to >> debug things. If you have a debugging version, I can test it. >> I think my regular expression code is broken in the gl stuff. Do you have to have the hardware and drivers for GL, or could I use an XML file for testing? The main thing I need to know is what the "name" of a GPU object really looks like. Perhaps the output of the hwloc gl test on an appropriate system would help. thanks, --Guy
Re: [hwloc-devel] python bindings testing help needed
On 06/04/2013 03:52 AM, Brice Goglin wrote: > (forgot to CC the list) > > > Le 04/06/2013 10:48, Brice Goglin a écrit : >> Hello, >> >> Here are some slightly fixed tests. cuda/cudart/nvml look good. >> intel-mic looks good but I couldn't test on a real machine (I used XML >> instead), so the cpuset retrieving code wasn't tested. >> >> gl doesn't seem to find any GPU, but I don't know enough about Python to >> debug things. If you have a debugging version, I can test it. >> >> By the way, I couldn't find a "libnuma" python package so I used "numa" >> in hwloc.py instead (that's what PIP was able to install). I have python-libnuma at git://git.fedorahosted.org/python-libnuma.git or http://people.redhat.com/streeter/ I should have included a reminder about that. >> >> Are you working on OpenCL too? The hwloc_opencl_* functions are static function definitions in the opencl.h file. I'd have to implement them in python instead of writing python interfaces to the library routines, and I'd need python bindings for the opencl libraries in order to do that. --Guy >> >> Brice >> >> >> >> Le 03/06/2013 23:16, Guy Streeter a écrit : >>> I'm getting the hwloc-1.7 version of the python bindings ready to go. I've >>> tried to add GL, Intel MIC, NVML, and CUDA support. >>> I can't implement much of CUDA or NVML for lack of python bindings for those >>> libraries, but I did what I could. >>> I can't test any of these for lack of hardware. If you have a Linux system >>> with some appropriate hardware and you can help, please either: >>> >>> clone the git repo at >>> git://git.fedorahosted.org/python-hwloc.git >>> and then checkout the 1.7-devel branch >>> >>> or >>> >>> get the tarball from >>> http://people.redhat.com/streeter/python-hwloc-1.7.tar.bz2 >>> and unpack it. >>> >>> Export the PYTHONPATH environment variable to point to the top level >>> directory, then run the various tests, like: >>> >>> python tests/gl.py >>> >>> also cuda.py, cudart.py, intel_mic.py, nvml.py >>> >>> As I said, the CUDA and NVML tests are only partly implemented. >>> All the other tests in the tests directory should match the output of the >>> same >>> test in the hwloc 1.7 source tree. >>> >>> Let me know how it goes. >>> thanks, >>> --Guy >>> ___ >>> hwloc-devel mailing list >>> hwloc-de...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel >
[hwloc-devel] python bindings testing help needed
I'm getting the hwloc-1.7 version of the python bindings ready to go. I've tried to add GL, Intel MIC, NVML, and CUDA support. I can't implement much of CUDA or NVML for lack of python bindings for those libraries, but I did what I could. I can't test any of these for lack of hardware. If you have a Linux system with some appropriate hardware and you can help, please either: clone the git repo at git://git.fedorahosted.org/python-hwloc.git and then checkout the 1.7-devel branch or get the tarball from http://people.redhat.com/streeter/python-hwloc-1.7.tar.bz2 and unpack it. Export the PYTHONPATH environment variable to point to the top level directory, then run the various tests, like: python tests/gl.py also cuda.py, cudart.py, intel_mic.py, nvml.py As I said, the CUDA and NVML tests are only partly implemented. All the other tests in the tests directory should match the output of the same test in the hwloc 1.7 source tree. Let me know how it goes. thanks, --Guy
Re: [hwloc-devel] nvidia and nouveau driver differences
On 05/03/2013 04:13 PM, Brice Goglin wrote: > If I remember correctly, NVIDIA or AMD proprietary drivers cannot use > the kernel sysfs API because it's GPL-only. They can't create devices in > sysfs, that's why hwloc doesn't get any GPU OS device with NVIDIA. > That sounds right and makes sense. > card* and controlD64 is what we get with opensource DRM drivers that use > the sysfs/drm kernel API. But I don't expect people to do much with them > as long as there's no way for an application to know if it's using card0 > or card1. That's why there's a NVIDIA specific plugin using NVCtrl: you > give a display such as :0.0, it returns the locality of the PCI device > running it. > > Brice Can you give me an example of something that should show the display device when the Nvidia driver is loaded? I think I properly configured hwloc: - Hwloc optional build support status (more details can be found above): Probe / display I/O devices: PCI GL Graphical output (Cairo):yes XML input / output: full libnuma memory support: yes Plugin support: no - thanks, --Guy
[hwloc-devel] nvidia and nouveau driver differences
The short answer to this may be that my Nvidia driver is too old. My Nvidia card is old so I have to use the version 173 proprietary driver on it, but the latest nouveau driver works fine. I'm on Fedora 18, testing hwloc-1.7. I fetched and built the latest libXNVCtrl, 319.17, from ftp://download.nvidia.com/XFree86/nvidia-settings. The one shipped with Fedora is too old and hwloc will not build with it. When I have the nvidia proprietary driver installed, it does not show as an OS device. Here's the end of the tests/hwloc_iodevs output: Found 3 OS devices Found OS device em1 subtype 2 Found OS device sda subtype 0 Found OS device sr0 subtype 0 with the open-source nouveau driver loaded, it looks like this: Found 5 OS devices Found OS device card0 subtype 1 Found OS device controlD64 subtype 1 Found OS device em1 subtype 2 Found OS device sda subtype 0 Found OS device sr0 subtype 0 The card0 and controlD64 objects have no infos associated with them. There are no other differences in the output. Both drivers seem operational. I've attached the gather-topology output (created with the nvidia driver loaded) in case anyone wants to see it. thanks, --Guy mentos.hsv.redhat.com.tar.bz2 Description: application/bzip
[hwloc-devel] manpage / --help discrepancies
"hwloc-assembler-remote --help" shows a --force option not mentioned in the man-page. The man page documents a --verbose option not recognized by the command. "hwloc-distrib --help" does not mention the --top option described in the man page. thanks, --Guy
[hwloc-devel] python bindings updated to 1.6
I have finally made the 1.6 additions to python-hwloc. userdata and xml import/export od userdata works. I spent some time trying to find a way to let userdata be a reference to a python object, but in the end decided to leave associating the userdata value with data in python space up to the user. Where in a C program you might make userdata a pointer to some structure or data area, in python you would need to put your data in a list or integer-keyed dictionary, and store the list index or dictionary key in the userdata field. As always, the git repo is at http://git.engineering.redhat.com/git/users/streeter/streeter-pyhwloc.git and rpm/srpm packages and source tarball are at http://people.redhat.com/streeter/ --Guy
Re: [hwloc-devel] CPU affinity of OS Devices?
On 11/06/2012 05:20 PM, Brice Goglin wrote: > Le 06/11/2012 23:55, Guy Streeter a écrit : >> On 11/06/2012 03:53 PM, Brice Goglin wrote: >>> Hello Guy, >>> >>> I don't think OS devices ever had a cpuset. All objects that are not >>> things where you can bind processes usually have NULL cpusets. So when >>> you have a PCI or OS device, you walk up the obj->parent pointer until >>> you find an object with a non-NULL cpuset. That's the affinity you're >>> looking for. >>> >>> You can use hwloc_get_non_io_ancestor_obj() (in hwloc/helper.h) to find >>> the first parent with non-NULL cpuset. >>> >>> Brice >>> >> I didn't mean to imply that they had gone away. My question is how do I >> specify a binding like "not on the same CPU that is handling the Ethernet >> interrupts"? >> > > Assuming you have the OS device for this interrupt, find the parent > object whose cpuset field isn't NULL, reverse this cpuset with > hwloc_bitmap_not() and bind to some object that is inside the resulting > cpuset. > > Brice > The problem with that is that it will exclude all of the CPUs associated with the node, or all of the CPUs on the machine if it isn't NUMA. The CPU affinity of the IRQ can be set independent of the bounding object of the PCI bridge the device is below. On my machine, the ahci device uses IRQ 90, and the affinity of that IRQ is CPU 3. The first parent with a cpuset above the OSDev block object for my disk drive is the machine itself. --Guy
Re: [hwloc-devel] CPU affinity of OS Devices?
On 11/06/2012 03:53 PM, Brice Goglin wrote: > Hello Guy, > > I don't think OS devices ever had a cpuset. All objects that are not > things where you can bind processes usually have NULL cpusets. So when > you have a PCI or OS device, you walk up the obj->parent pointer until > you find an object with a non-NULL cpuset. That's the affinity you're > looking for. > > You can use hwloc_get_non_io_ancestor_obj() (in hwloc/helper.h) to find > the first parent with non-NULL cpuset. > > Brice > I didn't mean to imply that they had gone away. My question is how do I specify a binding like "not on the same CPU that is handling the Ethernet interrupts"? thanks, --Guy
[hwloc-devel] CPU affinity of OS Devices?
I noticed on my system (Fedora 17) that the OS Devices don't have their own cpuset. It seems like it would be good to know the affinity of the interrupt assigned to the device. Is there a provision for this in hwloc, or would I need to find it another way? thanks, --Guy
[hwloc-devel] non-PCI OS devices?
When I'm trying to visualize how my system is physically configured, in order to decide how best to bind applications, I'd like to know about every source of interrupts. keyboard and mouse, timers, sound, USB-connected devices, etc. Has any though been given to expanding the OS device information to include them? thanks, --Guy
Re: [hwloc-devel] Why doesn't my disk controller show up?
On 10/09/2012 03:12 PM, Brice Goglin wrote: > Le 09/10/2012 22:12, Brice Goglin a écrit : >> The attached patch should help (and fix minor bugs nearby). >> >> Brice >> > > Patch attached for real. > > Yep, works fine. Thanks for the fast response! --Guy
Re: [hwloc-devel] Why doesn't my disk controller show up?
On 10/09/2012 02:18 PM, Guy Streeter wrote: > On 10/09/2012 02:03 PM, Brice Goglin wrote: >> Le 09/10/2012 20:47, Guy Streeter a écrit : >>> I'm trying to understand why for one of my systems the disk controller is >>> not >>> shown as an OS device. The PCI device is seen, but there is no OS device >>> under >>> it. It's a SATA controller emulated as SCSI, It think. >>> gather-topology output is attached. Using hwloc 1.5 on Fedora 17. >>> >> >> hwloc-gather-topology does not gather I/O devices unfortunately (we >> could gather OS devices, but we can't for PCI devices). >> >> What do you have under /sys/bus/pci/devices/ ? >> Do you see anything like hostX in there? What does in contain? Can you >> walk in there until you find a "block" child? >> >> I haven't tested on recent kernels lately, maybe something changed in >> the sysfs hierarchy, I am booting a 3.5 to check things. >> >> Brice >> >> ___ >> hwloc-devel mailing list >> hwloc-de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel >> > > There are entries like that. I've attached the output of > find -P /sys/bus/pci/devices/\:00\:1f.2/ > > thanks, > -Guy Oops. I needed a / on the end. New output attached --Guy /sys/bus/pci/devices/:00:1f.2/ /sys/bus/pci/devices/:00:1f.2/irq /sys/bus/pci/devices/:00:1f.2/ata1 /sys/bus/pci/devices/:00:1f.2/ata1/host0 /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0 /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/scan /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/unchecked_isa_dma /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/control /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/async /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/runtime_enabled /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/runtime_active_kids /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/runtime_active_time /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/autosuspend_delay_ms /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/runtime_status /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/runtime_usage /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/power/runtime_suspended_time /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/state /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/cmd_per_lun /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/sg_tablesize /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/em_message_type /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/active_mode /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/sg_prot_tablesize /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/unique_id /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/supported_mode /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/em_message_supported /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/device /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/ahci_host_cap2 /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/ahci_host_caps /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/subsystem /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/ahci_port_cmd /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/host_reset /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/uevent /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/proc_name /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/em_message /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/prot_capabilities /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/can_queue /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/ahci_host_version /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/prot_guard_type /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/em_buffer /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/link_power_management_policy /sys/bus/pci/devices/:00:1f.2/ata1/host0/scsi_host/host0/host_busy /sys/bus/pci/devices/:00:1f.2/ata1/host0/target0:0:0 /sys/bus/pci/devices/:00:1f.2/ata1/host0/target0:0:0/power /sys/bus/pci/devices/:00:1f.2/ata1/host0/target0:0:0/power/control /sys/bus/pci/devices/:00:1f.2/ata1/host0/target0:0:0/power/async /sys/bus/pci/devices/:00:1f.2/ata1/host0/target0:0:0/power/runtime_enabled /sys/bus/pci/devices/:00:1f.2/ata1/host0/target0:0:0/power/runtime_ac
Re: [hwloc-devel] Why doesn't my disk controller show up?
On 10/09/2012 02:03 PM, Brice Goglin wrote: > Le 09/10/2012 20:47, Guy Streeter a écrit : >> I'm trying to understand why for one of my systems the disk controller is not >> shown as an OS device. The PCI device is seen, but there is no OS device >> under >> it. It's a SATA controller emulated as SCSI, It think. >> gather-topology output is attached. Using hwloc 1.5 on Fedora 17. >> > > hwloc-gather-topology does not gather I/O devices unfortunately (we > could gather OS devices, but we can't for PCI devices). > > What do you have under /sys/bus/pci/devices/ ? > Do you see anything like hostX in there? What does in contain? Can you > walk in there until you find a "block" child? > > I haven't tested on recent kernels lately, maybe something changed in > the sysfs hierarchy, I am booting a 3.5 to check things. > > Brice > > ___ > hwloc-devel mailing list > hwloc-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel > There are entries like that. I've attached the output of find -P /sys/bus/pci/devices/\:00\:1f.2/ thanks, -Guy /sys/bus/pci/devices/:00:1f.2
[hwloc-devel] Why doesn't my disk controller show up?
I'm trying to understand why for one of my systems the disk controller is not shown as an OS device. The PCI device is seen, but there is no OS device under it. It's a SATA controller emulated as SCSI, It think. gather-topology output is attached. Using hwloc 1.5 on Fedora 17. thanks, --Guy baby.hsv.redhat.com.tar.bz2 Description: application/bzip
[hwloc-devel] python bindings updated for hwloc 1.5 (also, python-libnuma is available)
I have updated python-hwloc to support version 1.5 of hwloc. It is available at git://git.fedorahosted.org/python-hwloc.git as well as rpm packages and source tarball on http://people.redhat.com/streeter/ In the course of this, I added the linux_libnuma calls. python-hwloc now depends on python-libnuma, which is also available on my people page and at git://git.fedorahosted.org/python-libnuma.git I suppose it's time I admitted I haven't made any effort to make sure python-hwloc works on non-Linux systems. If you want to use it on another OS, let me know of problem you encounter and I'll try to fix them. --Guy
Re: [hwloc-devel] lstopo-nox strikes back
On 04/25/2012 04:38 AM, Brice Goglin wrote: > Hello, > > We recently got some complains from redhat/centos users that wanted to install > hwloc on their cluster but couldn't because it brought so many X libraries > that they don't care about. > > Debian solves this by having two hwloc packages: the main hwloc one, and > hwloc-nox where cairo is disabled. You just install one of them, packages are > marked as conflicting with each others. > > I asked Jirka, our fellow RPM hwloc packager. He feels that RPM distros don't > work that way. They usually have a core 'foo' package without X, and something > such as 'foo-gui' with the X-enabled binary. So you'd have lstopo and > lstopo-gui installed at the same time. > > I don't have any preference but RPM is much more widely used than deb in HPC, > so we must consider the issue, either in hwloc or in RPM packaging. And we > need a solution that is consistent across distros (we don't want users to get > lost because Debian/Ubuntu lstopo is graphical while RPM lstopo is not and > lstopo-gui is). > > It's not hard to build two lstopo binaries in the same hwloc (quick patch > attached). But we'd need to decide their names (lstopo/lstopo-nox, > lstopo/lstopo-nogui, lstopo-gui/lstopo), and find a good way to make the > existing packages deal with them. > > How do people feel about this? Is it ok to choose between hwloc and hwloc-nox > packages on Debian/Ubuntu? Does somebody want to *always* have a lstopo-nox > installed? Should the default lstopo be graphical/cario or not? > > Brice > Fedora generally prefers separate packages with separate commands, like system-config-network-tui and system-config-network. Red Hat addressed the problem in the "tuna" package for the realtime product by simply removing the graphical package dependencies and making tuna run in commandline mode if the attempt to start graphical mode fails. The downside to that approach is that you have to figure out for yourself what dependencies to install if you want the graphical output. It happens, though, that they are standard packages that are installed for most any system with a desktop environment. --Guy
Re: [hwloc-devel] understanding PCI device to NUMA node connection
On 11/28/2011 03:45 PM, Brice Goglin wrote: ... > Current Intel platforms have 2 QPI links going to I/O hubs. Most servers > with many sockets (4 or more) thus have each I/O hub connected to only 2 > processors directly, so their distance is "equal" as you say. > > However, some BIOS report invalid I/O locality information. I've never > seen anything correct on any server like the above actually. If the BIOS correctly reported the locality, how would the devices show up in hwloc-info and lstopo? Would there be a Group containing the 2 NUMA Nodes and the PCI devices? > > Yes, unfortunately PCI detection isn't based on reading files, so > there's no easy way to "dump" it during gather-topology.sh. I knew this once, and remembered it when you explained it. thanks, --Guy
[hwloc-devel] understanding PCI device to NUMA node connection
This question may be more about understanding NUMA (which I barely do) than about hwloc, but perhaps you can help anyway. I have a customer with some HP Proliant DL580 G7 servers. HP supplied them with a block diagram of their system, and it shows two of the NUMA nodes connected to the PCI devices through an I/O Hub. The customer thinks hwloc ought to show the PCI devices associated with both of the NUMA nodes. I'm not sure how that's possible. hwloc shows them all under the first node. Is the association of the devices with the nodes correct? Can the devices actually be equally "close" to both of them? On a side note, hwloc-gather-topology apparently doesn't gather device information? I got the output from their system but can't see any devices when I use it as input to hwloc-info etc. thanks, --Guy
Re: [hwloc-devel] memory corruption problem with libpci
On 11/03/2011 12:51 PM, Guy Streeter wrote: > I have a RHEL6.1 system on which I get a glibc error report: > > *** glibc detected *** hwloc-1.3/bin/lstopo: double free or corruption > (!prev): 0x01945630 *** > === Backtrace: = > /lib64/libc.so.6[0x3d1a275716] > /lib64/libpci.so.3[0x3d1aa06bdf] > /lib64/libpci.so.3(pci_free_dev+0x1e)[0x3d1aa02c9e] > /lib64/libpci.so.3(pci_cleanup+0x20)[0x3d1aa029e0] > /root/hwloc-1.3/src/.libs/libhwloc.so.4(+0x1e8e2)[0x7f3ef14908e2] > /root/hwloc-1.3/src/.libs/libhwloc.so.4(+0xa1fe)[0x7f3ef147c1fe] > /root/hwloc-1.3/src/.libs/libhwloc.so.4(hwloc_topology_load+0x1a2)[0x7f3ef147d136] > hwloc-1.3/bin/lstopo[0x405050] > /lib64/libc.so.6(__libc_start_main+0xfd)[0x3d1a21ec9d] > hwloc-1.3/bin/lstopo[0x402d49] It appears to be a libpci bug: https://bugzilla.redhat.com/show_bug.cgi?id=740630 fix is expected to ship in RHEL6.2 thanks, --Guy
[hwloc-devel] memory corruption problem with libpci
I have a RHEL6.1 system on which I get a glibc error report: *** glibc detected *** hwloc-1.3/bin/lstopo: double free or corruption (!prev): 0x01945630 *** === Backtrace: = /lib64/libc.so.6[0x3d1a275716] /lib64/libpci.so.3[0x3d1aa06bdf] /lib64/libpci.so.3(pci_free_dev+0x1e)[0x3d1aa02c9e] /lib64/libpci.so.3(pci_cleanup+0x20)[0x3d1aa029e0] /root/hwloc-1.3/src/.libs/libhwloc.so.4(+0x1e8e2)[0x7f3ef14908e2] /root/hwloc-1.3/src/.libs/libhwloc.so.4(+0xa1fe)[0x7f3ef147c1fe] /root/hwloc-1.3/src/.libs/libhwloc.so.4(hwloc_topology_load+0x1a2)[0x7f3ef147d136] hwloc-1.3/bin/lstopo[0x405050] /lib64/libc.so.6(__libc_start_main+0xfd)[0x3d1a21ec9d] hwloc-1.3/bin/lstopo[0x402d49] I tried writing a test program that does the same kinds of things with libpci that I think hwloc does, but I cannot reproduce this outside of hwloc. Does it look familiar? How can I help debug this? thanks, --Guy
Re: [hwloc-devel] final v1.3?
On 10/11/2011 01:59 AM, Brice Goglin wrote: Anybody against releasing the final v1.3? We only had the following changes since v1.3rc2. Nothing worth a new tarball? * Fix lstopo crash in verbose messages about offline/disallowed/... processors when root has no cpuset, just don't display anything for now * Update HWLOC_DOXYGEN_BROKEN_SHORT_NAMES doc Makefile fix * Use hwloc_snprintf when the returned value matters. Fixes testsuite on hp-ux. * Pass -w to diff to avoid CRLF vs LF difference on Windows. We had about 50 visits on the v1.3 download page since rc2 (and more than 300 since rc1). I hope people did actually test it. I have been testing it from svn quite a bit, as I prepare the python bindings to support it. I have no issues with it. --Guy Brice ___ hwloc-devel mailing list hwloc-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel
Re: [hwloc-devel] hwloc trunk nightly 1.3a1r3511 fails to build on CentOS 5.6 & RHEL 5.6
pciutils was re-based to 3.1.7 in RHEL5.6 at the request of some of our hardware partners, in order to add some new hardware support. That'd explain why I didn't hit it when I was testing previously, we were still on CentOS 5.4 and RHEL 5.5 then so obviously this dependency is new in RHEL 5.6. Wonder if they'd consider it a bug ? Apparently not. The only bug filed about it (that I have found) is BZ 646884, which reported that kudzu failed to build after the update. The response to that was, "add -lresolv" --Guy
[hwloc-devel] python bindings for hwloc 1.2
I have updated the python bindings to support hwloc-1.2. They are backward-compatible with 1.1. There is a Fedora rpm and a tarball on http://people.redhat.com/streeter/ and the git archive is git://git.fedorahosted.org/python-hwloc.git ssh://git.fedorahosted.org/git/python-hwloc.git http://git.fedorahosted.org/git/python-hwloc.git --Guy
[hwloc-devel] set_synthetic documentation error
The man page and the online documentation for hwloc_topology_set_synthetic() both say that it takes a comma-separated list as input. It appears to take a space-separated list instead. Also, the documentation does not indicate what the return value means. When the input description cannot be parsed, the routine returns -1 and does not set errno. thanks, --Guy
[hwloc-devel] multiple simultaneous topology inits?
I was considering the design of an administration tool in python, and my original idea was to have each module that needs the topology object create its own. This means that different parts of the same program (but possibly in different threads) would call init and load on their own topology structure, and use that structure in their hwloc routines calls. Is this a safe/reasonable/efficient way to go, or should I call init and load on only one structure and pass that one around throughout the application? thanks, --Guy
[hwloc-devel] arbitrary length limit in attr_snprintf
The output of hwloc_obj_attr_snprintf() is arbitrarily limited. In particular, the infos portion is at most 255 characters long. On my systems there is quite a bit more in the infos for the machine object. --Guy
[hwloc-devel] typo in hwloc-bind.c
else if (!strncmp(argv[1], "interleace", 2)) should be else if (!strncmp(argv[1], "interleave", 2)) --Guy
Re: [hwloc-devel] git repo for python bindings
On 01/21/2011 10:48 AM, Guy Streeter wrote: The python bindings are now available at http://git.fedorahosted.org/git/python-hwloc.git or git://git.fedorahosted.org/python-hwloc.git --Guy I have significantly updated the python bindings implementation: Renamed the classes; Topology, Obj, Bitmap, etc. Removed the unnecessary hwloc_ prefix everywhere. (physically, in the source code) separated the helpers from the API and gathered them by the doxygen groups in helpers.h Implemented more iterators. Corrected a few bugs. Made sure that all of the hwloc_obj except userdata is readonly. Changed the accessors for bitmaps in the hwlog_obj structure so they always return a copy of the bitmap. This prevent accidental modification of the const data. Made a few enhancements to my python implementation of lstopo. Implemented hwloc-distrib and hwloc-calc in python. Other things I can't think of right now. --Guy
[hwloc-devel] git repo for python bindings
The python bindings are now available at http://git.fedorahosted.org/git/python-hwloc.git or git://git.fedorahosted.org/python-hwloc.git --Guy
Re: [hwloc-devel] python bindings
On 01/20/2011 01:22 PM, Samuel Thibault wrote: Bernd Kallies, le Thu 20 Jan 2011 20:13:03 +0100, a écrit : The only thing I was wondering about are these OO method names, which I had to invent them myself. If there is some need to synchronize these with the python binding, we may agree about them. Backwards compatibility of the Perl binding is not a real problem, since there may be many aliases for a function in the base XS code. Could you perhaps list those you have defined already, if any? I added some iterators: bitmap.all_set_bits obj.infos topology.objs_by_depth topology.objs_by_type I also made obj.children an iterator. I think I could do the same with siblings and cousins if that makes sense. I changed all the snprintf functions or asprintf methods. I implemented bitmap.bitmap_weight() as a method, but also bitmap.weight as a property. Also bitmap.bitmap_first() and bitmap.first, etc. I'm thinking I could do away with the method calls and just provide the properties. Instead of the bitmap isequal, or, and, xor, and not methods, I supported the operators ==, != |, |=, &, &=, ^. ^=, and ~. A new topology instance gets init() automatically, and is destroyed when it goes away, so those methods are not exposed.
Re: [hwloc-devel] python bindings
On 01/20/2011 11:32 AM, Samuel Thibault wrote:
Hello,
Guy Streeter, le Mon 17 Jan 2011 21:03:04 +0100, a écrit :
I am currently working to get a public git repository set up so that I can
share the work. In the meantime, my first pass at python bindings for hwloc
are available from
http://people.redhat.com/streeter/
Here are some comments on some bits of code:
topo = hwloc.hwloc_topology()
assert obj.type == hwloc.HWLOC_OBJ_PU
orig = hwloc.hwloc_bitmap.alloc()
Mmm, why repeating "hwloc"?
I think removing the "hwloc_" prefix from the class names is a good idea. I'll
do that.
Do you think I should also do it for those constants? I thought matching the C
constant name was better, but hwloc.OBJ_PU is less typing :)
Also, since they are class names, do you think I should capitalize them as is
common for python classes? "topo = hwloc.Topology()" ?
It's nice to have rewritten the whole testsuite :) It's really useful to
have a very good grasp on how it looks like.
hwlocset = topo.complete_cpuset.dup()
Mmm, so if the user forgets to use dup(), he might still be able to
write in a const cpuset?
I haven't figured out a way to mark things as non-modifiable. I'm still
researching it. Right now nothing stops you changing any bitmap.
def set_thread_cpubind(self, tid, cpuset):
"""Bind a thread on cpus given in cpuset"""
return _linux_set_thread_cpubind(self, tid, cpuset)
Mmm, no. If you use _linux_set_thread_cpubind, then call the python
function linux_set_thread_cpubind, as this is using TIDs, which is a
Linux invention. The normal set_thread_cpubind function should use the
usual python thread identifier (just like hwloc's set_thread_cpubind
uses the usual C thread identifier: pthread_t). I see in some other code
that you wonder how to create pthread in python. Then don't provide the
function for now (rather than providing a function which does not take
the proper thread identification type).
OK, I understand this now.
Also, could you clearly separate functions which are not defined
in hwloc.h itself? In the C interface, they are in a separate e.g.
helpers.h file in order to express that these are really helpers only,
and that the user doesn't need to know them all, everything can be done
with the core functions from hwloc.h.
Do you mean physically separate in the source code, or are you talking about
providing a different way to access them? I know the source code could use
some cleanup and more comments.
Also, for more clarity, maybe you should also separate functions by
comments according to the doxygen groups in hwloc.h, see \defgroup
hwlocality_api_version API version which only contains HWLOC_API_VERSION
and hwloc_get_api_version for now, for instance. That should make
review and maintenance quite easier.
assert b2.ulong() == 0xa0a
I wonder whether ulong() and ulong(i) are useful for the python
bindings: these are mostly useful for legacy interfaces which just use
single integers. Python has unbound integers, so it's not really useful
in that case :) One thing that might be useful however is b2.uint32()
and b2.uint64(), I guess we should provide it at the C stage first.
I couldn't think of a use for them either. I tried to implement everything
that is documented in the man pages. There may be other things that don't make
sense.
alloc_membind doesn't make a lot of sense in python, since you really
can't tell the python interpreter to use the space.
I can't think of a way to use hwloc_set_area_membind* in python
They should probably take a python object itself, but since I guess the
python GC does what it wants with moving data... That might however
be useful when e.g. somebody drives C computation code from a python
fast-prototyping main loop.
_libhwloc = ctypes.CDLL(ctypes.util.find_library('hwloc'),
use_errno=True)
Mmm, shouldn't this include the soname of the library? Else built
bindings will break on ABI changes.
I planned to check the library version at run-time. That way I can provide
backward compatibility.
Apart from these nasty details, I like the interface style, thanks for
the nice contribution ! :)
b1 = hwloc.hwloc_bitmap.alloc('0xf')
b2 = hwloc.hwloc_bitmap.alloc(range(4))
Yay :)
I'm adding more support for bitmap representations:
assert b3.intersects('0xf...f,0xfffd,0xfff9')
assert b1.compare_first(0x3)
assert b1.compare('3') < 0
--Guy
Re: [hwloc-devel] python bindings
On 01/18/2011 06:47 AM, Jeff Squyres wrote: Are the abstractions anything like Bernd's perl bindings, perchance? http://search.cpan.org/~bka/ On Jan 17, 2011, at 3:02 PM, Guy Streeter wrote: I am currently working to get a public git repository set up so that I can share the work. In the meantime, my first pass at python bindings for hwloc are available from http://people.redhat.com/streeter/ There are rpm files and a source tarball there. The tarball has implementations of some of the tests and utils in python. It assumes hwloc-1.1 and python-2.6 or later. Let me know what you think. --Guy ___ hwloc-devel mailing list hwloc-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel I am not familiar enough with Perl to examine the source and say if the implementations are similar. In my python bindings (like all of python) everything is an object. Anything that requires the topology is a method or property of the topology object. Things that just need a hwloc_obj are methods or properties of the hwloc_obj object. Same for bitmaps. It tried to make things easy and obvious. There is an iterator for children of an object, and one for set bits in a bitmap, so you can do for child in obj.children: or for idx in bitmap: You can compare bitmaps with the equality operator, you can use the bitwise & and | operators on them as well. A bitmap used in string context will get a string representation. I made asprintf instead of snprintf methods because allocating a string buffer to write into doesn't make sense in python. Objects are freed when they go out of context, in cases where that is appropriate. There are some python-specific test files in the tests directory, which I tried to comment as examples. --Guy
[hwloc-devel] python bindings
I am currently working to get a public git repository set up so that I can share the work. In the meantime, my first pass at python bindings for hwloc are available from http://people.redhat.com/streeter/ There are rpm files and a source tarball there. The tarball has implementations of some of the tests and utils in python. It assumes hwloc-1.1 and python-2.6 or later. Let me know what you think. --Guy
[hwloc-devel] documentation/implementation questions with the snprintf routines
The man pages for the various snprintf routines indicate that they return the number of characters written. My testing indicates that they always return the number of characters required, which can be more or less than the space supplied. My testing also indicates that if the buffer size is indicated as zero, the buffer pointer is not referenced. This allows me to pass in NULL and zero to determine the required size before allocating the space. I would like to see that explicitly promised so I can count on it going forward. thanks, --Guy
Re: [hwloc-devel] Is hwloc_bitmap_from_ith_ulong working?
On 12/22/2010 01:50 PM, Guy Streeter wrote:
I'm using 1.1 on Fedora 13. I start with
set = hwloc_bitmap_alloc_full()
and it looks like
0xf...f
Then I call
hwloc_bitmap_from_ith_ulong(set, 1, 0xdeadbeef)
and I get
0xdeadbeef,0x0034,0x3f77aed8
which I don't understand.
Example code is attached.
thanks,
--Guy
hwloc_bitmap_set_ith_ulong() seems to behave in a similar way.
full: 0xf...f
ith: 0xf...f,,0xdeadbeef,0x,0x
Example attached.
thanks,
--Guy
#include
#include
#include
main()
{
hwloc_bitmap_t set;
char *string;
set = hwloc_bitmap_alloc_full();
hwloc_bitmap_asprintf(&string, set);
printf("full: %s\n", string);
free(string);
hwloc_bitmap_set_ith_ulong(set, 1, 0xdeadbeef);
hwloc_bitmap_asprintf(&string, set);
printf("ith: %s\n", string);
return 0;
}
[hwloc-devel] Is hwloc_bitmap_from_ith_ulong working?
I'm using 1.1 on Fedora 13. I start with
set = hwloc_bitmap_alloc_full()
and it looks like
0xf...f
Then I call
hwloc_bitmap_from_ith_ulong(set, 1, 0xdeadbeef)
and I get
0xdeadbeef,0x0034,0x3f77aed8
which I don't understand.
Example code is attached.
thanks,
--Guy
#include
#include
#include
main()
{
hwloc_bitmap_t set;
char *string;
set = hwloc_bitmap_alloc_full();
hwloc_bitmap_asprintf(&string, set);
printf("full: %s\n", string);
free(string);
hwloc_bitmap_from_ith_ulong(set, 1, 0xdeadbeef);
hwloc_bitmap_asprintf(&string, set);
printf("ith: %s\n", string);
return 0;
}
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2975
On 12/22/2010 09:02 AM, Brice Goglin wrote:
Le 22/12/2010 15:59, Guy Streeter a écrit :
On 12/22/2010 02:36 AM, Brice Goglin wrote:
I'll backport this in 1.1 later, in case somebody wants to fix the
English speaking.
Brice
For the benefit of the Perl and Python bindings efforts, it would be
good if we could query the runtime library for it's API version.
Ah, right, we forgot about this. Something like this?
unsigned hwloc_get_api_version(void)
{
return HWLOC_API_VERSION;
}
We can put this in 1.1.1. But how will you handle earlier releases?
Brice
That would be great. In Python at least, I can determine at runtime that the
hwloc_get_api_version symbol is not in the library. And I'm targeting 1.1 for
my efforts, so I can just document that as the minimal working version.
thanks,
--Guy
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2975
On 12/22/2010 02:36 AM, Brice Goglin wrote: I'll backport this in 1.1 later, in case somebody wants to fix the English speaking. Brice For the benefit of the Perl and Python bindings efforts, it would be good if we could query the runtime library for it's API version. --Guy Le 22/12/2010 09:28, bgog...@osl.iu.edu a écrit : Author: bgoglin Date: 2010-12-22 03:28:55 EST (Wed, 22 Dec 2010) New Revision: 2975 URL: https://svn.open-mpi.org/trac/hwloc/changeset/2975 Log: Add a FAQ entry about HWLOC_API_VERSION Text files modified: trunk/doc/hwloc.doxy |36 1 files changed, 36 insertions(+), 0 deletions(-) Modified: trunk/doc/hwloc.doxy == --- trunk/doc/hwloc.doxy(original) +++ trunk/doc/hwloc.doxy2010-12-22 03:28:55 EST (Wed, 22 Dec 2010) @@ -1294,5 +1294,41 @@ (that may for instance be transmitted between applications through a socket). +\section faq_upgrade How do I handle API upgrades? + +The hwloc interface is extended with every new major release. +Any application using the hwloc API should be prepared to check at +compile-time whether some features are available in the currently +installed hwloc distribution. + +To check whether hwloc is at least 1.1, you should use: +\verbatim +#include +#if HWLOC_API_VERSION>= 0x00010100 +... +#endif +\endverbatim
Re: [hwloc-devel] 1.0.3rc3 and 1.1rc5
On 12/20/2010 12:56 PM, Brice Goglin wrote: Le 20/12/2010 19:40, Guy Streeter a écrit : Get this singlethreaded process memory : expected 0x000f, got 0xf...f Is that a bug? That's on my Fedora 13 non-numa system. This is kind of expected. 0x000f means all the cores in the machine. 0xf...f means all the machine when the machine isn't numa. It's hard to make things strictly the same without loosing some semantics and keeping things simple. Fortunately, your application won't see any actual difference unless it actually compares full cpusets like above. On my RHEL6.0 numa system, Get this singlethreaded process memory : expected 0x00ff, got 0x003f In which section of hwloc_bind is this? Sometimes, the difference is normal too because Linux doesn't exactly do what we want. Brice ___ hwloc-devel mailing list hwloc-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel I decided I should just give you the whole output. --Guy [root@amd-istanbul-24 tests]# ./hwloc_bind system set is 0x00ff Bind this singlethreaded process: OK Get this singlethreaded process: OK Bind this thread: OK Get this thread: OK Bind this whole process : OK Get this whole process : OK Bind whole process : OK Get whole process : OK Bind process: OK Get process: OK Bind thread : OK Get thread : OK now strict Bind this singlethreaded process: OK Get this singlethreaded process: OK Bind this thread: OK Get this thread: OK Bind this whole process : OK Get this whole process : OK Bind whole process : OK Get whole process : OK Bind process: OK Get process: OK Bind thread : OK Get thread : OK obj set is 0x003f Bind this singlethreaded process: OK Get this singlethreaded process: OK Bind this thread: OK Get this thread: OK Bind this whole process : OK Get this whole process : OK Bind whole process : OK Get whole process : OK Bind process: OK Get process: OK Bind thread : OK Get thread : OK now strict Bind this singlethreaded process: OK Get this singlethreaded process: OK Bind this thread: OK Get this thread: OK Bind this whole process : OK Get this whole process : OK Bind whole process : OK Get whole process : OK Bind process: OK Get process: OK Bind thread : OK Get thread : OK singlified to 0x0001 Bind this singlethreaded process: OK Get this singlethreaded process: OK Bind this thread: OK Get this thread: OK Bind this whole process : OK Get this whole process : OK Bind whole process : OK Get whole process : OK Bind process: OK Get process: OK Bind thread : OK Get thread : OK now strict Bind this singlethreaded process: OK Get this singlethreaded process: OK Bind this thread: OK Get this thread: OK Bind this whole process : OK Get this whole process : OK Bind whole process : OK Get whole process : OK Bind process: OK Get process: OK Bind thread : OK Get thread : OK memory tests complete node set i.e. cpuset 0x00ff default Bind this singlethreaded process memory : OK Get this singlethreaded process memory : OK Bind this thread memory : OK Get this thread memory : OK Bind this whole process memory : XFAILED (38, Function not implemented) Get this whole process memory : XFAILED (38, Function not implemented) Bind process memory : XFAILED (38, Function not implemented) Get process memory : XFAILED (38,
Re: [hwloc-devel] 1.0.3rc3 and 1.1rc5
On 12/20/2010 12:56 PM, Brice Goglin wrote: Le 20/12/2010 19:40, Guy Streeter a écrit : Get this singlethreaded process memory : expected 0x000f, got 0xf...f Is that a bug? That's on my Fedora 13 non-numa system. This is kind of expected. 0x000f means all the cores in the machine. 0xf...f means all the machine when the machine isn't numa. It's hard to make things strictly the same without loosing some semantics and keeping things simple. Fortunately, your application won't see any actual difference unless it actually compares full cpusets like above. On my RHEL6.0 numa system, Get this singlethreaded process memory : expected 0x00ff, got 0x003f In which section of hwloc_bind is this? Sometimes, the difference is normal too because Linux doesn't exactly do what we want. complete node set i.e. cpuset 0x00ff ... bound Bind this singlethreaded process memory : OK Get this singlethreaded process memory : expected 0x00ff, got 0x003f Bind this thread memory : OK Get this thread memory : expected 0x00ff, got 0x003f happens in the 'migrate' section also, but not the strict ones. --Guy
Re: [hwloc-devel] 1.0.3rc3 and 1.1rc5
On 12/16/2010 06:03 AM, Jeff Squyres wrote: Both have been posted. http://www.open-mpi.org/software/hwloc/v1.0/ http://www.open-mpi.org/software/hwloc/v1.1/ I have not updated the docs for these rc's on the web site in anticipation that these are final/quick rc's and the real release will be sometime later today. Please test so that we can do the final release today. Sorry for being slow on this. 1.1rc5, the hwloc_bind test gives me messages like Get this singlethreaded process memory : expected 0x000f, got 0xf...f Is that a bug? That's on my Fedora 13 non-numa system. On my RHEL6.0 numa system, Get this singlethreaded process memory : expected 0x00ff, got 0x003f thanks, --Guy
Re: [hwloc-devel] Hwloc perl binding
On 12/15/2010 02:02 PM, Brice Goglin wrote: ... As I said, we're trying to keep things consistent. My understanding is that a function should have a topology parameter if it traverses some links between objects inside the topology. If it only operates on a given object and does not traverse the topology at all, it will likely not need any topology parameter. I noticed that some functions (in or not in the ABI I don't recall right now, I'm not at my desk) need the topology parameter only because the topology structure holds pointers to platform-specific routines. That seems like a confusing generalization of the parameter's meaning/purpose. It would be helpful, when creating bindings for object-oriented languages, if for instance operations on bindings could be methods of the bindings object, without the complication of holding the topology pointer in each object instance. --Guy
Re: [hwloc-devel] SWIG bindings
On 11/30/2010 10:52 AM, Jeff Squyres wrote: On Nov 30, 2010, at 11:44 AM, Guy Streeter wrote: Would anyone object if I take a whack at making some SWIG bindings for hwloc? I'm thinking specifically for perl (because that's my scripting language of choice), but I could probably be convinced to look at python as well. I'm currently working on a python bindings package for hwloc. I suppose I should find out how to get my code upstream. Great! Did you use SWIG, or do it manually? I am doing this manually. I think SWIG works fine for simple interfaces, but this is pretty complicated. I have enough of it working that I re-wrote hwloc-hello in python. I'm going to write python versions of all the tests next. --Guy
Re: [hwloc-devel] SWIG bindings
On 11/30/2010 10:07 AM, Jeff Squyres wrote: Would anyone object if I take a whack at making some SWIG bindings for hwloc? I'm thinking specifically for perl (because that's my scripting language of choice), but I could probably be convinced to look at python as well. (this would be for 1.2 at the earliest -- definitely not for 1.1) I'm currently working on a python bindings package for hwloc. I suppose I should find out how to get my code upstream. --Guy
[hwloc-devel] python bindings for libhwloc?
Has anyone worked on or expressed interest in python bindings for the hwloc library? I do most of my work in python and would find it useful. I threw together a python implementation of the hwloc-hello program in python, using ctypes to access the library. It was fairly straight-forward. I would be happy to work on a bindings package if it is not being done by someone else. thanks, --Guy
