yes, and it can be assumed that some modules that are imported in yade could be
the reason fort he problem:[1]
"After some more googling I found the answer here.
It turns out that certain Python modules (numpy, scipy, tables, pandas,
skimage...) mess with core affinity on import. As far as I can tell, this
problem seems to be specifically caused by them linking against multithreaded
OpenBLAS libraries.
A workaround is to reset the task affinity using
os.system("taskset -p 0xff %d" % os.getpid())
With this line pasted in after the module imports, my example now runs on all
cores.
There are also two ways to disable the CPU affinity-resetting behaviour of
OpenBLAS itself. At run-time you can use the environment variable
OPENBLAS_MAIN_FREE (or GOTOBLAS_MAIN_FREE), for example
OPENBLAS_MAIN_FREE=1 python myscript.py
Or alternatively, if you're compiling OpenBLAS from source you can permanently
disable it at build-time by editing the Makefile.rule to contain the line
NO_AFFINITY=1
"
Are we compiling OpenBLAS from source?
But even if we are, why does the cpu affinity problem appear even for a built
from late 2013 which definitely worked well concerning multicore operation?
Maybe it was an update oft he openblas modul that was done by apt-get update?
[1]
http://stackoverflow.com/questions/15639779/what-determines-whether-different-python-processes-are-assigned-to-the-same-or-d?lq=1
answer#25
-----Ursprüngliche Nachricht-----
Von: Yade-dev
[mailto:yade-dev-bounces+alexander.eulitz=iwf.tu-berlin...@lists.launchpad.net]
Im Auftrag von Christian Jakob
Gesendet: Freitag, 28. Februar 2014 10:52
An: Yade Development Group
Betreff: [Yade-dev] cpu affinity problem
Hi there,
I just tried to find out what is not working at my machine in parallel mode
(using yade -jN). First thing I recognized is different MHz for my CPUs are
used:
me@debian ~/YADE/build >less /proc/cpuinfo | grep MHz
cpu MHz : 3167.68
cpu MHz : 2000.0
cpu MHz : 2000.0
cpu MHz : 2000.0
cpu MHz : 2000.0
cpu MHz : 2000.0
cpu MHz : 2000.0
cpu MHz : 2000.0
I disabled GV1/GV3 and C mode for CPU performance in BIOS:
(I skipped the part where I had to reinstall kde and graphic driver because PC
was running for 276 days and there was a kernel update, which broke my graphics
module, which leads to black screen when starting kdm, which .... whatever ... )
me@debian ~/YADE/build >less /proc/cpuinfo | grep MHz
cpu MHz : 3158.704
cpu MHz : 3158.704
cpu MHz : 3158.704
cpu MHz : 3158.704
cpu MHz : 3158.704
cpu MHz : 3158.704
cpu MHz : 3158.704
cpu MHz : 3158.704
But still the same problem occurs (for new and old versions of yade).
So I googled that and it seems to be a problem with cpu affinity.
When I run yade -j4 --performance I see 4 PIDs (threads) running at 1 and the
same core the whole time (e.g. PID = {11962,11963,11964,11965} in htop).
Then I tried this ...
me@debian ~/YADE/trunk >taskset -c -p 0 11962 pid 11962's current affinity
list: 0 pid 11962's new affinity list: 0 me@debian ~/YADE/trunk >taskset -c -p
1 11963 pid 11963's current affinity list: 0 pid 11963's new affinity list: 1
me@debian ~/YADE/trunk >taskset -c -p 2 11964 pid 11964's current affinity
list: 0 pid 11964's new affinity list: 2 me@debian ~/YADE/trunk >taskset -c -p
3 11965 pid 11965's current affinity list: 0 pid 11965's new affinity list: 3
and viola, it is working as expected (at least until one run with X balls is
finished, after that PID is changing again and all new PIDs switched back on
CPU 0).
So my question is:
How can I set cpu affinity? Do you know a fast-and-dirty way to get rid of this?
Thanks in advance,
cheesed off Christian
_______________________________________________
Mailing list: https://launchpad.net/~yade-dev
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yade-dev
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~yade-dev
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yade-dev
More help : https://help.launchpad.net/ListHelp
