On Tue, Sep 26, 2006 at 04:24:34PM -0400, Denis Papathanasiou wrote: > >If you're not reading/writing the file (via with-open-file or open > >or whatever) then why lock it? If you *are* reading/writing the > >file, why not use that file descriptor? > > The requirements are a bit unique: we have two different > applications (two distinct processes) launched at different times, > via a cron scheduler (this is running on a linux server, btw). > > Those two processes read from and write to the same set of files. > > Even though they've been scheduled to run at different times (to > avoid running on top of each other), it *is* possible (though given > the schedule unlikely), that those two processes will try to access > the same file at the same time.
You feel victim to one of the classic blunders, the most famous of which is "Never get involved in a land war in Asia", but only slightly less well known is this: Never schedule two related cronjobs and hope they don't step on each other. Put them both in a script to run serially and schedule the script. > >Your description makes me think you've implemented an interface > >conceptually similar to this: > > > > (defun lock (fname) > > (let ((fd (open ...))) > > (when fd > > (fcntl fd LOCK)))) > > > > (defun unlock (fname) > > (let ((fd (open ...))) > > (when fd > > (fcntl fd UNLOCK)))) > > > >and that just won't work. > > You're right, the moment you exit either of those functions, the > lock is gone. Well, it's wrong, but not in that way. Neither function closes the file handle, but they don't return it either, so nobody else can close it, so they "leak" file handles. But if you're not doing it that way, nevermind. :) > Here's how we did it instead -- this is still just the first > attempt, subject to change, but it's passed all the tests we've done > so far: > > (defun with-file-lock (file-pathname fn &rest args) > "Perform file I/O on file-pathname (i.e. apply fn and its args), > while setting a POSIX fcntl() lock on file-pathname to prevent > simultaneous writes/race conditions." > (let ((fd -1) > (result nil)) > (loop > (progn > (setf fd (lockfile (namestring file-pathname))) > (when (> fd -1) > (unwind-protect > (setf result (apply fn args)) > (unlockfile fd)) > (return-from with-file-lock result))) > (sleep 0.2)))) > > So basically, we attempt to place a lock on the file designated by the > file-pathname object, but if we can't (because the other process has > it), we pause and try again. > > Once we do get a lock, and the fd is valid, we apply fn (whatever > read/write action we need to do on the file), remove the lock, and > return. > > Note that (lockfile) and (unlockfile) correspond to the fcntl() > wrapper classes in lockfile.o, as defined by (alien:def-alien-routine) > after we load the object file using (alien:load-foreign). > > So that's a long way of saying: I think we're doing everything > correctly now, but I know how tricky these things can be (hence the > long battery of tests we're putting it through), so if you have any > comments or suggestions, please let me know. That looks okay to my non-Lisp-expert eyes. I'd probably write it as a macro, but tastes (and requirements :) vary. Note that if FN raises a condition, with-file-lock will return NIL. Not necessarily an error, of course. -- L
