On Wed, Dec 27, 2000 at 08:49:51PM +1300, Dan Langille wrote:
> FreshPorts2 will have a new processing strategy for incoming
> messages. Each message will be in a separate file in a predetermined
> directory. As each file arrives, it is processed by a perl script. I want
> only one instance of that perl script running at a given time. This is
> primarily for serialization and to ensure the system doesn't get bogged
> down running perl scripts if many messages arrive in a short period of
> time.
>
> My idea is to have a daemon, or something resembling one, sitting on
> the box watching the directory. When a new file appears, it starts a perl
> script. This perl script is beyound the scope of my question, but it
> processes all the files in the directory. When finished, it looks for any
> more files and repeats as necessary. If no more files, it exits.
>
> If a file arrives, the daemon checks to see if the perl script is already
> running. If so, it doesn't start another one.
>
> Any ideas on how to do this? Any suggestions on the process?
I would do that (and have done it in several projects) using opendir()
and readdir(). Open the directory, read entry by entry, when you find
a file you want, process it and unlink() it. Get to the end of the dir,
sleep, repeat.
Beware of a subtle problem here though - see that you do not have
the process which creates files creating them in that directory; you
might very well wind up with a file being processed before it's fully
created. There are two solutions to this problem - either DJB's
Maildir style, or processing files based on filenames.
DJB's Maildir concept is based on having two directories - a temporary
one where files are created and then atomically move/rename'd to
the real one. This works best when the tempdir and the real dir are
located on the same filesystem, and you can use the rename() syscall.
However, there is a solution if you want the temporary dir on another
filesystem - there is a safecat program, which I shall shortly commit
a port for (it's been sitting in my to-do tree for several weeks now).
The other way is create the files in the same directory, but with
a different name style, e.g. ending in .tmp; then when you readdir()
an entry, only process those not ending in .tmp, or only process those
ending in .xml, or something like that. This might be a bit easier
to implement.
G'luck,
Peter
--
If there were no counterfactuals, this sentence would not have been paradoxical.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
