Dear all,
I found a problem with openMPI jobs when I set the $TMP/$TMPDIR to an
nfs mounted storage. OpenMPI (version 1.8.4 with intel) writes out an
error message that it will create some files there which might slow
things down. And this is exactly what happens. I actually have the
executables in a D state instead of R and things are ridiculously slow.
The problem does not happen when I use qlogin and I found that
$TMP/$TMPDIR are not set and so I assume openMPI is using local /tmp
instead? Also, all is fine when I run directly on the nodes. Also,
things wehre fien before I switched SGE temp to the nfs file system.
I need the SGE temp directory to be the nfs mounted file system, so I
cannot change that. But does somebody know how I can tell openMPI to use
local /tmp?
Out of curiosity, why does a jobs via qsub set $TMPDIR and qlogin does
not? What is the difference between the two? I have been using the
qlogin for testing when things have gone wrong via qsub assuming that
the environment set up would be the same.
Best wishes and thank you in advance for your help
Marlies
--
ccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccms
Please note change of work hours: Monday, Wednesday and Friday
Dr. Marlies Hankel
Research Fellow
High Performance Computing, Quantum Dynamics& Nanotechnology
Theory and Computational Molecular Sciences Group
Room 229 Australian Institute for Bioengineering and Nanotechnology (75)
The University of Queensland
Qld 4072, Brisbane
Australia
Tel: +61 (0)7-33463996
Fax: +61 (0)7-334 63992
mobile:+61 (0)404262445
Email: [email protected]
http://web.aibn.uq.edu.au/cbn/
ccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccmsccms
Notice: If you receive this e-mail by mistake, please notify me, and do
not make any use of its contents. I do not waive any privilege,
confidentiality or copyright associated with it. Unless stated
otherwise, this e-mail represents only the views of the Sender and not
the views of The University of Queensland.
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
