It is more a NFS issue, if you have a file in NFS that two users in two different host are accessing (one at least write to it) you will need to enforce the "sync" option. Even if you flush all the data and close the file the NFS client can still have cached data that it didn't sync to the server.
Do you know if is applies to linux O_DIRECT writes as well? From comment in fs/nfs/direct.c: * When an application requests uncached I/O, all read and write requests * are made directly to the server; data stored or fetched via these * requests is not cached in the Linux page cache. The client does not * correct unaligned requests from applications. All requested bytes are * held on permanent storage before a direct write system call returns to * an application. -- mg
