On Wed, Jul 22, 2009 at 17:17, Sylvain Jeaugey<sylvain.jeau...@bull.net> wrote: > Hi Jeff, > > I'm interested in joining the effort, since we will likely have the same > problem with SLURM's cpuset support. > > On Wed, 22 Jul 2009, Jeff Squyres wrote: > >> But as to why it's getting EINVAL, that could be wonky. We might want to >> take this to the PLPA list and have you run some small, non-MPI examples to >> ensure that PLPA is parsing your /sys tree properly, etc. > > I don't see the /sys implication here. Can you be more precise on which > files are read to determine placement ? Most files under /sys/devices/system/cpu/cpu*/topology/*
> > IIRC, when you are inside a cpuset, you can see all cpus (/sys should be > unmodified) but calling set_schedaffinity with a mask containing a cpu > outside the cpuset will return EINVAL. No. The Linux kernel ands the cpumask of the tasks cpuset with the given affinity mask. If no cpuset is involved it takes the online mask. After that, it checks if the resulting mask is a subset of the online mask and would return -EINVAL. > The only solution I see to solve this > would be to get the "allowed" cpus with sched_getaffinity, which should be > set according to the cpuset mask. sched_getaffinity() doesn't return the cpuset mask. It returns the mask the task can run, which is a subset of the cpuset. Also the initial mask of the task (right after exec) does not to be the cpuset mask, because the affinity mask is inherited from the parent. I does not know any C interface to get a tasks cpuset mask (ok, libcpuset, looks like this lib is debian now, note to myself: check this). The Cpus_allowed* fields in /proc/<pid>/status are the same as sched_getaffinity returns and the /proc/<pid>/cpuset needs to be resolved, i.e. where is the cpuset fs mounted? Bert > > Sylvain > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
