https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64719
Bug ID: 64719
Summary: omp_get_num_procs should not account for cpu affinity
Product: gcc
Version: 4.9.2
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: libgomp
Assignee: unassigned at gcc dot gnu.org
Reporter: geir at cray dot com
CC: jakub at gcc dot gnu.org
Created attachment 34525
--> https://gcc.gnu.org/bugzilla/attachment.cgi?id=34525&action=edit
omp.c
The description for the routine omp_get_num_procs is stated as:
Returns the number of processors online on that device.
The GCC implementation behavior of omp_get_num_procs does not follow this
description, but rather does the following:
Returns the number of processors that have affinity to the calling
process/thread.
The Cray and PGI versions of omp_get_num_procs will return the actual number of
processors available on the device without regard to affinity. [I suppose there
could be some interpretation what is meant by the term "online", but then this
should be further explained in the description of omp_get_num_procs.]
Our job launcher will set affinity to optimize performance of parallel programs
(MPI, SHMEM, Gloab Arrays, UPC, Coarray Fortran, ...). In the following
example two processes that have three OpenMP threads are being invoked:
GCC compiled program:
$ export OMP_NUM_THREADS=3
$ aprun -n 2 -d 3 ./omp.gcc
Sysconf get nprocs reports 24 processors
Sysconf get nprocs reports 24 processors
Hello from pid 14544, thread 0 (core affinity = 0) (Available OpenMP processors
= 1)
Hello from pid 14544, thread 2 (core affinity = 2) (Available OpenMP processors
= 1)
Hello from pid 14544, thread 1 (core affinity = 1) (Available OpenMP processors
= 1)
Hello from pid 14545, thread 0 (core affinity = 3) (Available OpenMP processors
= 1)
Hello from pid 14545, thread 2 (core affinity = 5) (Available OpenMP processors
= 1)
Hello from pid 14545, thread 1 (core affinity = 4) (Available OpenMP processors
= 1)
$
Cray compiler compiled program:
$ export OMP_NUM_THREADS=3
$ aprun -n 2 -d 3 ./omp.cray
Sysconf get nprocs reports 24 processors
Sysconf get nprocs reports 24 processors
Hello from pid 14620, thread 2 (core affinity = 2) (Available OpenMP processors
= 24)
Hello from pid 14620, thread 0 (core affinity = 0) (Available OpenMP processors
= 24)
Hello from pid 14620, thread 1 (core affinity = 1) (Available OpenMP processors
= 24)
Hello from pid 14621, thread 0 (core affinity = 3) (Available OpenMP processors
= 24)
Hello from pid 14621, thread 2 (core affinity = 5) (Available OpenMP processors
= 24)
Hello from pid 14621, thread 1 (core affinity = 4) (Available OpenMP processors
= 24)
$
Here is the GCC compiled version with CPU affinity turned off:
$ aprun -cc none -n 2 -d 3 ./omp.gcc
Sysconf get nprocs reports 24 processors
Sysconf get nprocs reports 24 processors
Hello from pid 15044, thread 1 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15044, thread 2 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15044, thread 0 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15045, thread 2 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15045, thread 0 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15045, thread 1 (core affinity = 0-23) (Available OpenMP
processors = 24)
$
----------------------
I realize that this bug will very likely be closed as INVALID or WONTFIX, but
the documentation needs to be fixed to make clear the behavior of
omp_get_num_procs.
