[trimming the Cc: list -- note Postfix is also affected, list address [EMAIL PROTECTED], subscription via regular majordomo interface]
On Wed, 15 Jan 2003, Sam Varshavchik wrote: > Matthias Andree writes: > > >In case you've missed it, I'll repeat my suggestion: I suggested keeping > >the tmp/timestamp.pid.hostname files for a finite amount of time after > >linking them to new/ (so that for 20 seconds or 2 seconds or what > >timeout you choose, both file names exist) -- the tmp/ is then a kind of > > Who's gonna remove them? > > You want qmail-local, or whatever delivers the message, to sleep around, > with its thumb up its ass for 2-20 seconds, before unlinking the tmp file, > and exiting? No, because then we could go for Henning's "ugly" idea of sleeping before exiting (or before delivering, to ward off collisions when the process crashes during sleep), a cleanup daemon would have to be used indeed, writing file names + timestamp with O_APPEND and one write per name to tmp/.deletelist.hostname would be safe I presume. OTOH, the other suggestion (inode and all that) are saner and simpler IMO, so they'd get my vote instead of this "keep tmp files around" thingy. > It doesn't matter how many CPUs the machine has. If microseconds are added > to the filename, you have to turn things around in less than a microsecond > for the race to remain (I might've messed up my units a few times in the > last couple of days, but I'm always referring to tv_usec's, one millionth > of a second). This _will_ happen. > I actually had another brainstorm today. Add the file's inode, when moving > the message from tmp to new: > > tmp/timestamp.unique_pid.hostname -> new/timestamp.inode.unique_pid.hostname > How does that grab you? It does add an extra stat() call, to obtain the Optimization: use fstat() before closing the tmp/* file, and make an additional fsync() after the link() before letting go of the file descriptor with close(), to cater for softupdates on BSD (note it's the default for /var on fresh installs of the recent FreeBSD releases, but let's not discuss this further in this thread, split this out with new Subject if you like). > message's inode, after the file is created in tmp. Looks good, and after I brought this topic up on Postfix users, Wietse suggested renaming the tmp/time.pid_count.hostname file to new/time.pid_inode.hostname, where inode is the inode number taken from fstat() on the open tmp/time.pid_count.hostname file descriptor. Archive links: (same message) http://marc.theaimsgroup.com/?l=postfix-users&m=104268277911943&w=2 http://msgs.securepoint.com/cgi-bin/get/postfix0301/369/1.html Wietse pointed out this will still give rise to collisions if the new/* files are then moved to a different file system. Victor Duchovni then followed up with another different suggestion, namely new/time.pid_hash.hostname. Archive link: http://marc.theaimsgroup.com/?l=postfix-users&m=104268602714321&w=2 Quoting Victor: "At some noticeable performance cost using new/SHA-1(file content) would make all the problems go away. For a maildir holding 10^6 messages you get a collision probability of ~ 10^12/10^(0.3*160) = 10^-36. This is far below the probability of undetected disk CRC errors losing the data." -- Matthias Andree ------------------------------------------------------- This SF.NET email is sponsored by: Thawte.com Understand how to protect your customers personal information by implementing SSL on your Apache Web Server. Click here to get our FREE Thawte Apache Guide: http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0029en _______________________________________________ courier-users mailing list [EMAIL PROTECTED] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
