Re: [OMPI devel] [RFC] Hierarchical Topology

2010-11-15 Thread Ralph Castain
Guess I am a little confused. Every MPI process already has full knowledge of what node all other processes are located on - this has been true for quite a long time. Once my work is complete, mpirun will have full knowledge of each node's hardware resources. Terry will then use that in mpirun's m

Re: [OMPI devel] [RFC] Hierarchical Topology

2010-11-15 Thread Sylvain Jeaugey
I already mentionned it answering Terry's e-mail, but to be sure I'm clear : don't confuse node full topology with MPI job topology. It _is_ different. And every process does not get the whole topology in hitopo, only its own, which should not cause storms. On Mon, 15 Nov 2010, Ralph Castain

Re: [OMPI devel] [RFC] Hierarchical Topology

2010-11-15 Thread Sylvain Jeaugey
On Mon, 15 Nov 2010, Terry Dontje wrote: A few comments: 1. Have you guys considered using hwloc for level 4-7 detection? Yes, and I agree there may be something to improve on level 4-7 detection. But note that hitopo differs from hwloc because it is not discovering the whole machine, only w

Re: [OMPI devel] [RFC] Hierarchical Topology

2010-11-15 Thread Ralph Castain
I think the two efforts (the paffinity and this one) do overlap somewhat. I've been writing the local topology discovery code for Jeff, Terry, and Josh - uses hwloc (or any other method - it's a framework) to discover what hardware resources are available on each node in the job so that the info ca

Re: [OMPI devel] [RFC] Hierarchical Topology

2010-11-15 Thread Terry Dontje
A few comments: 1. Have you guys considered using hwloc for level 4-7 detection? 2. Is L2 related to L2 cache? If no then is there some other term you could use? 3. What do you see if the process is bound to multiple cores/hyperthreads? 4. What do you see if the process is not bound to any

[OMPI devel] [RFC] Hierarchical Topology

2010-11-15 Thread Sylvain Jeaugey
As a followup of Stuttgart's developper's meeting, here is an RFC for our topology detection framework. WHAT: Add a framework for hardware topology detection to be used by any other part of Open MPI to help optimization. WHY: Collective operations or shared memory algorithms among others may