On Tue, 2011-03-01 at 11:00 +0100, Brice Goglin wrote: > Le 01/03/2011 10:46, Bernd Kallies a écrit : > > To calculate topology-based pinning schemes and do process pinning (like > > done e.g. by OpenMPI or MVAPICH2) this is too long, when every process > > (MPI task) or thread loads the topology in parallel. But exporting an > > XML topology and using this for this purpose is inaccepteable, when > > Linux cpusets are used, because one needs the topology of a subset of > > the machine depending on the caller context. What we currently do is to > > let only one process per machine load the topology, and distribute the > > essentials needed for pinning to the remaining processes. > > > > You can still export one XML file per cpuset and load the right one in > each process. If the batch scheduler creates the cpuset, my feeling is > that the batch scheduler should create the XML file at the same time, > and do something like export HWLOC_XMLFILE=mycpuset.xml when launching > processes.
Yes, that's a good idea, as long as the extent of the cpuset is not modified after the batch scheduler created it (e.g. because it is not aware of SMT, or because the scheduler does not handle node access policies correctly). Things are in progress regarding this for Torque, so letting it create a HWLOC_XMLFILE is definitely worth to implement. > Also, in 1.2, we'll have a hwloc_topology_restrict() function which will > let you load the whole machine topology and then restrict it to whatever > part of it (a part is defined by a hwloc_cpuset_t). We'll need to make > sure that you'll have everything you need to get your cpuset's > hwloc_cpuset_t. Thanks for the info. Such a thing is definitely needed. BK > Brice > > _______________________________________________ > hwloc-devel mailing list > [email protected] > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel -- Dr. Bernd Kallies Konrad-Zuse-Zentrum für Informationstechnik Berlin Takustr. 7 14195 Berlin Tel: +49-30-84185-270 Fax: +49-30-84185-311 e-mail: [email protected]
