On 10/01/10 01:14, Giuseppe Scrivano wrote:
when --all fails for any reason, I think we should try with the number
of available processing units, at least it is a more accurate value than
return 1 (and document this behaviour).
Bruno, Jim, what do you think?
Just to summarize what's happening here...
There are 3 CPU counts possible:
total >= online >= available
"online" corresponds to the CPUs enabled system wide,
whereas "available" is what's available to a particular
process which may be less due to affinity config.
"online" is not currently exposed through nproc
but for reference is got on linux using:
$ strace -e open getconf _NPROCESSORS_ONLN
open("/proc/stat"
for_each_online_cpu() //available to scheduler
"available" on linux is determined using:
nproc
glibc::sched_getaffinity()
return corresponding_syscall();
"total" on linux is determined using:
nproc --all
glibc::sysconf(_NPROCESSORS_CONF)
__get_nprocs_conf ()
{
int result = 1;
/* XXX Here will come a test for the new system call. */
if (open("/sys/devices/system/cpu"))
result = parse();
else if (open("/proc/cpuinfo"))
result = parse();
return result;
}
The last one above is giving the issue as
it relies on /sys or /proc being available.
The XXX comment above suggests that there might
be a new syscall to get the total count, but
`man syscalls` doesn't show anything obvious to me.
Now we don't get a failure from sysconf(_NPROCESSORS_CONF),
so really that needs to be fixed. I think in the unusual
case where one can't open /proc/ or /sys/ that it should
try sched_getaffinity() as that will be at least as
accurate as hard coding "1" and will maintain the
expected count ordering above.
I will log a bug against glibc for this.
So what about a possible work around?
How about doing this in nproc(1):
if (num_processors(NPROC_ALL) == 1)
return num_processors(NPROC_CURRENT_OVERRIDABLE)
That would of course do a redundant syscall in certain cases
but that might be OK actually as relative to the cost
of a process it's not much, and --all wouldn't
be the usual usage anyway.
The alternative of adding skips to the test seem
a bit more messy to me.
Proposed fix is attached.
cheers,
Pádraig.
>From 9eb8bf9a70a1390f894844020a5ff6ae74209a89 Mon Sep 17 00:00:00 2001
From: =?utf-8?q?P=C3=A1draig=20Brady?= <p...@draigbrady.com>
Date: Sun, 10 Jan 2010 02:04:34 +0000
Subject: [PATCH] nproc: return a possibly more accurate total CPU count
GNU/Linux systems that don't have /sys/ or /proc/ mounted
may return "1" for the total CPU count. Incur the overhead of
the sched_getaffinity syscall in this case as though not
accurate may be more accurate than "1". This addresses a
misc/nproc-avail test failure noticed by Dmitry V. Levin.
---
src/nproc.c | 7 +++++++
1 files changed, 7 insertions(+), 0 deletions(-)
diff --git a/src/nproc.c b/src/nproc.c
index 8092df7..d7b0305 100644
--- a/src/nproc.c
+++ b/src/nproc.c
@@ -116,6 +116,13 @@ main (int argc, char **argv)
nproc = num_processors (mode);
+ /* On certain GNU/Linux systems where /sys/ or /proc/ are not mounted,
+ * num_processors (NPROC_ALL) will always return 1. So in this case always
+ * call using NPROC_CURRENT_OVERRIDABLE as that uses a syscall
+ * to determine the count, and so may be more accurate in this case. */
+ if (mode == NPROC_ALL && nproc == 1)
+ nproc = num_processors (NPROC_CURRENT_OVERRIDABLE);
+
if (ignore < nproc)
nproc -= ignore;
else
--
1.6.2.5
