Sean Kelly wrote:
dsimcha wrote:
== Quote from Sean Kelly ([email protected])'s article
dsimcha wrote:
Is there an easy way to write a function, or a function already
written for
me, that will allow for a file shared between processes to be
appended to
safely? This should be done in a way that will make it impossible
for two
processes to write to it at the same time, using locking, and should be
platform-independent.
Performance does not matter because, in the use case I have, we're only
talking about one update every few minutes. Simplicity, however,
does matter.
All I'm trying to do is run a simulation thousands of times on a
bunch of
different computers sharing an NFS file system and have all of the
results end
up in one nice plain text file instead of having each instance write
to its
own file and having to keep track of them and piece them together by
hand.
Use a second file to represent the lock, with the contents as the lock
owner. While the file exists, poll. When it's not there, create it and
write the process id into it, then unlink the file when you're done.
Wouldn't you have to somehow atomically poll and create the file?
What if some
other process created the lock file between your call of exists() and
write()?
If you use the "create only" flag when opening the file then it should
fail if the file was already created by someone else. Unless NFS
doesn't provide a sufficiently reliable synchronization mechanism for
this to work, that is (I really don't know).
I've worked a lot with NFS and have the scars and learned the curses to
prove it.
NFS is as non-deterministic as it gets when it comes about concurrent
writes. There is next to no guarantee. The append problem is an absolute
classic on NFS. I tried about five different schemes, all failed under
mysterious circumstances. What I do now and suggest you do too is to
have each different process create its own file. After all processes
have ended, have a master process assemble all small files into one.
It's really the only thing I got to work.
Andrei
