Yes, on ext3, an fsync() syncs the entire filesystem. It has to, because
all the metadata for each file is shared - it's just a string of
journallable blocks. Similar story with the data, in ordered mode.
So effectively, fsync()ing five files one time each is performing 25 fsync()s.
One fix (which makes the application specific to ext3 in ordered-data or
journalled-data mode) is to perform a single fsync(), with the
understanding that this has the side-effect of fsyncing all the other
files. That's an ugly solution and is rather hard to do if the workload
consists of five separate processes!
So I'd recommending mounting the filesystem with the `-o data=writeback'
mode. This way, each fsync(fd) will sync fd's data only. That's much
better than the default data-ordered mode, wherein a single fsync() will
sync all the other file's data too.
In data=writeback mode it is still the case that fsync(fd) will sync the
other file's metadata, but that's a single linear write to the journal and
the additional cost should be low.
Bottom line: please try data=writeback, let me know.
Does this mean that those of us using XFS should run some testing as well?
thanks,
joshua
----
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
