On Mon, 8 Jun 2009, Rainer Gerhards wrote: >> -----Original Message----- >> From: [email protected] [mailto:rsyslog- >> [email protected]] On Behalf Of [email protected] >> >> On Mon, 8 Jun 2009, Rainer Gerhards wrote: >> >>> Hi all, >>> >>> I am working on the ultra-reliable file writing capability. I have > revisited >>> the doc and am considering fsync() vs. fdatasync(). From the man pages, > it >>> looks like fdatasync() is sufficient if I just need to ensure that any > data >>> written can also be read. The advantage of fdatasync() is that it is >> somewhat >>> faster as not all meta data is necessarily written. >>> >>> I would appreciate if someone could comment on this issue. >> >> my understanding is that if you are growing the file you need to do a >> fsync of the directory, but if you are re-writing part of an already >> allocated file fdatasync is the better thing to use. > > The man page[1] is somewhat vague. It tells: > > "Calling fsync() does not necessarily ensure that the entry in the directory > containing the file has also reached disk. For that an explicit fsync() on a > file descriptor for the directory is also needed." > > But also > > "fdatasync() is similar to fsync(), but does not flush modified metadata > unless that metadata is needed in order to allow a subsequent data retrieval > to be correctly handled. For example, changes to st_atime or st_mtime > (respectively, time of last access and time of last modification; see > stat(2)) do not not require flushing because they are not necessary for a > subsequent data read to be handled correctly. On the other hand, a change to > the file size (st_size, as made by say ftruncate(2)), would require a > metadata flush. > " > > So my current understanding is that fdatasync() is always safe, but there > still exist situations where a sync on the directory is needed. What I can > think of is a newly created file.
any time the file size changes (so that the metadata about the disk allocation is safe) > I have to admit that I am a bit puzzled of what it takes to do the "explicit > fsync on the directory". Most importantly, I need to have the file handle > open for writing, else the fsync will potentially fail. I don't think this is > such an excellent idea on directories.. I am right now working on this, and > my search has not brought up many good hints. So what I currently plan to do > is an open(dirname, O_RDWR) on the directory and then do an fsync on it > later. I believe that that's correct. > And I guess even after the fsync() on the directeroy, there potentially is an > issue with newly created files inside newly create directories, but I don't > think I will address this immediately. But I may be useful/necessary to add a > tree walker who actually walks the full path name and calls the fsyncs on all > of them... in an ideal world yes, but in practice you would only need to do this when you create a new file and that new directory entry caused the filesystem to allocate more space on disk to hole the directory entry, so it's almost never an issue. another advantage of fdatasync is that it tells the OS exactly what portion of the file it needs to work on, with a fsync it needs to search all buffers related to that file to figure out what needs to be written (not a big hit, but a hit) David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
