Nathaniel Smith <n...@pobox.com> added the comment: That stackoverflow thread points to the GNU coreutils 'nproc', which is an interesting compendium of knowledge about this problem.
It looks like their algorithm is roughly: 1. Determine how many CPUs *could* this program access, by going down a list of possible options and using the first that works: pthread_getaffinity_np -> sched_getaffinity -> GetProcessAffinityMask -> sysconf(_SC_NUMPROCESSORS_ONLN) -> some arcane stuff specific to HP-UX, IRIX, etc. 2. Parse the OMP_NUM_THREADS and OMP_THREAD_LIMIT envvars (this is not quite trivial, there's some handling of whitespace and commas and references to the OMP spec) 3. If OMP_NUM_THREADS is set, return min(OMP_NUM_THREADS, OMP_THREAD_LIMIT). Otherwise, return min(available_processors, OMP_THREAD_LIMIT). Step (1) handles both the old affinity APIs, and also the cpuset system that docker uses. (From cpuset(7): "Cpusets are integrated with the sched_setaffinity(2) scheduling affinity mechanism".) Step (2) relies on the quasi-standard OMP_* envvars to let you choose something explicitly. The PBS Pro docs say that they set both NPROCS and OMP_NUM_THREADS. See section 6.1.7 of https://pbsworks.com/pdfs/PBSUserGuide14.2.pdf So this seems like a reasonable heuristic approach to me. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32986> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
