But its a very bad idea on a "coherent", "POSIX" filesystem like Lustre.
Locks have to bounce around between the nodes for every write. This can
be VERY slow (even for trivial amounts of "logging" IO) and thrash the
filesystem for other users. So, yes, at our site, we include this sort
of "parallel IO" on our list of disallowed behaviour. Not a good practice
to adopt in general.
David
On 02/03/2010 10:40 AM, Laurence Marks wrote:
I know it's wrong, but I don't think it is forbidden.... (which I
guess is what you are saying).
On Tue, Feb 2, 2010 at 5:31 PM, Jeff Squyres<jsquy...@cisco.com> wrote:
+1 on Nick's responses.
AFAIK, if you don't mind getting garbage in the output file, it should be fine
to do. Specifically: it should not cause OS issues (crash, reboot, corrupted
filesystem, etc.) to do this -- but the file contents will likely be garbage.
That being said, this situation likely falls into the "Doc, it hurts when I do
this..." category. Meaning: you know it's wrong, so you probably shouldn't be doing
it anyway. :-)
On Feb 2, 2010, at 4:50 PM, Nicolas Bock wrote:
Hi Laurence,
I don't know whether it's as bad as a deadly sin, but for us parallel writes
are a huge problem and we get complete garbage in the file. Take a look at:
Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI One-Sided
Communication, Robert Latham,Robert Ross, Rajeev Thakur, International Journal
of High Performance Computing Applications, 21, 132 (2007).
They describe an implemenation of a "mutex" like object in MPI. If you protect
writes to the file with an exclusive lock you can serialize the writes and make use of
NFS's close to open cache coherence.
nick
On Tue, Feb 2, 2010 at 08:27, Laurence Marks<l-ma...@northwestern.edu> wrote:
I have a question concerning having many processors in a mpi job all
write to the same file -- not using mpi calls but with standard
fortran I/O. I know that this can lead to consistency issues, but it
can also lead to OS issues with some flavors of nfs.
At least in fortran, there is nothing "wrong" with doing this. My
question is whether this is "One of the Seven Deadly Sins" of mpi
programming, or just frowned on. (That is, it should be OK even if it
leads to nonsense files, and not lead to OS issues.) If it is a sin, I
would appreciate a link to where this is spelt out in some "official"
document or similar.
--
Laurence Marks
Department of Materials Science and Engineering
MSE Rm 2036 Cook Hall
2220 N Campus Drive
Northwestern University
Evanston, IL 60208, USA
Tel: (847) 491-3996 Fax: (847) 491-7820
email: L-marks at northwestern dot edu
Web: www.numis.northwestern.edu
Chair, Commission on Electron Crystallography of IUCR
www.numis.northwestern.edu/
Electron crystallography is the branch of science that uses electron
scattering and imaging to study the structure of matter.
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Jeff Squyres
jsquy...@cisco.com
