Re: [O-MPI devel] Linux processor affinity

2005-12-14 Thread Jeff Squyres
On Dec 13, 2005, at 1:45 PM, Jeff Squyres wrote: PLPA should be available Real Soon Now. We have released v0.9 of the Portable Linux Processor Affinity (PLPA -- pronounced "pli-pa") project, a standalone library that hides all the muckety-muck of processor affinity that we have been discus

Re: [O-MPI devel] Linux processor affinity

2005-12-13 Thread Jeff Squyres
On Dec 13, 2005, at 12:30 PM, Bogdan Costescu wrote: In short -- we ended up agreeing with you. :-) Whew, I'm surprised given the enthusiasm which you showed when seeing Paul's code ! Really, I thought that you will choose Paul's code with only the conditions changed as expressed in my last e

Re: [O-MPI devel] Linux processor affinity

2005-12-13 Thread Bogdan Costescu
On Mon, 12 Dec 2005, Jeff Squyres wrote: In short -- we ended up agreeing with you. :-) Whew, I'm surprised given the enthusiasm which you showed when seeing Paul's code ! Really, I thought that you will choose Paul's code with only the conditions changed as expressed in my last e-mail... a

Re: [O-MPI devel] Linux processor affinity

2005-12-12 Thread Jeff Squyres
To make this significantly easier, I called Paul and we discussed this at length. In short -- we ended up agreeing with you. :-) As a personal sidenote -- it sucks that we all had to do this much research to figure this out. In particular, we missed the fact that all the kernel versions

Re: [O-MPI devel] Linux processor affinity

2005-12-12 Thread Bogdan Costescu
On Fri, 9 Dec 2005, Paul H. Hargrove wrote: If one looks though enough kernel versions, In the meantime, I've gotten a copy of kernel/sched.c from a SGI Prism kernel - I assume that it is the same used on Altix; this one has in the Makefile EXTRAVERSION = -sgi306rp31. So again, all prototype

Re: [O-MPI devel] Linux processor affinity

2005-12-09 Thread Paul H. Hargrove
Just recently finished checking. For the collection of Linux hosts I have access to, the probe results are the same regardless of the choice of set or get. I agree 100% that "get" is a safer probe. -Paul Jeff Squyres wrote: On Dec 9, 2005, at 3:06 PM, Bogdan Costescu wrote: rc = sche

Re: [O-MPI devel] Linux processor affinity

2005-12-09 Thread Jeff Squyres
On Dec 9, 2005, at 3:06 PM, Bogdan Costescu wrote: rc = sched_setaffinity(0, sizeof(mask), mask); This changes whatever affinity might have been set before this check, for example by a (smart, don't know if such exists now) batch system. I haven't checked if it's possible, but I think tha

Re: [O-MPI devel] Linux processor affinity

2005-12-09 Thread Bogdan Costescu
On Thu, 8 Dec 2005, Jeff Squyres wrote: This is friggen' amazing. Let me disagree with you here... and not because I proposed a different solution. ;-) rc = sched_setaffinity(0, sizeof(mask), mask); This changes whatever affinity might have been set before this check, for example b

Re: [O-MPI devel] Linux processor affinity

2005-12-09 Thread Paul H. Hargrove
If one looks though enough kernel versions, one finds that some of them differ in what they will accept for the len. Some produce EINVAL if len!=sizeof(long), others (especially Altix) produce EINVAL if len is too short to cover all the machine's CPUs. I think I recall finding one that was ev

Re: [O-MPI devel] Linux processor affinity

2005-12-09 Thread Bogdan Costescu
On Thu, 8 Dec 2005, Jeff Squyres wrote: Check out http://svn.open-mpi.org/svn/ompi/trunk/opal/mca/paffinity/ linux/paffinity_linux.h -- there's a big comment in that file about the problem, to include descriptions of the 3 APIs. I'm sorry, but that is not quite what I wrote about in my messag

Re: [O-MPI devel] Linux processor affinity

2005-12-08 Thread Jeff Squyres
On Nov 29, 2005, at 3:04 PM, Bogdan Costescu wrote: Here's the problem: there are 3 different APIs for processor affinity in Linux. Could you please list them (at least the ones that you know about) ? Check out http://svn.open-mpi.org/svn/ompi/trunk/opal/mca/paffinity/ linux/paffinity_linux

Re: [O-MPI devel] Linux processor affinity

2005-12-08 Thread Jeff Squyres
On Nov 29, 2005, at 2:51 PM, Paul H. Hargrove wrote: The result is the following, which I've tried in limited testing: Holy Crimminey, Batman -- this message slipped by me in my INBOX. This is friggen' amazing. Many thanks, Paul! enum { SCHED_SETAFFINITY_TAKES_2_ARGS, SCHED_SETAFFINI

Re: [O-MPI devel] Linux processor affinity

2005-11-29 Thread Bogdan Costescu
On Tue, 29 Nov 2005, Jeff Squyres wrote: Here's the problem: there are 3 different APIs for processor affinity in Linux. Could you please list them (at least the ones that you know about) ? In the kernel source, in kernel/sched.c, the sys_sched_setaffinity function appears only in 2.6.0 (tal

Re: [O-MPI devel] Linux processor affinity

2005-11-29 Thread Paul H. Hargrove
Eureka! Operationally the 3-argument variants are ALMOST identical. The older version required len == sizeof(long), while the later version allowed the len to vary (so an Altix could have more than 64 cpus). However, in the kernel both effectively treat the 3rd argument as an array of unsig

Re: [O-MPI devel] Linux processor affinity

2005-11-29 Thread Paul H. Hargrove
Jeff, et al., My own "research" into processor affinity for the GASNet runtime began by "borrowing" the related autoconf code from OpenMPI. My experience is the same as Jeff's when it comes to looking for a correlation between the API and any system parameter such as libc or kernel version

[O-MPI devel] Linux processor affinity

2005-11-29 Thread Jeff Squyres
Greetings all. I'm writing this to ask for help from the general development community. We've run into a problem with Linux processor affinity, and although I've individually talked to a lot of people about this, no one has been able to come up with a solution. So I thought I'd open this to