[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: Ronald Oussoren wrote: > Adding the F_FULLSYNC option to os.fsync would be fine though To show you that i'm not unteachable i'll attach a patch which does that. This approach can be heavily extended, then, e.g. by using sync_file_range(all_the_flags) on

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: 8-} The real message was: On Wed, Apr 20, 2011 at 12:16:39PM +, Antoine Pitrou wrote: > If Apple thinks [...] there's no reason for us Apple thinks (KernelProgramming.pdf, BPFileSystem.pdf): Kernel code must be nearly perfect. Kernel prog

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Wed, Apr 20, 2011 at 12:16:39PM +, Ronald Oussoren wrote: > This is the same behavior as fsync on OSX, and OSX also has > a second API that provides stronger guarantees. > With your patch it is no longer possible to call the C function > fsync on

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: On Wed, Apr 20, 2011 at 12:16:39PM +, Ronald Oussoren wrote: > This is the same behavior as fsync on OSX, and OSX also has > a second API that provides stronger guarantees. > With your patch it is no longer possible to call the C function > fsync on

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Charles-Francois Natali
Charles-Francois Natali added the comment: > in particular: linux doesn't guarantee that data gets writting to the disk > when you call fsync, only that the data gets pushed to the storage device. Barriers are now enable by default in EXT4, and Theodore Tso has been favourable to that for qui

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Antoine Pitrou
Antoine Pitrou added the comment: Agreed with Ronald. If Apple thinks fsync() shouldn't flush the hardware cache, there's no reason for us to override that. -- nosy: +pitrou ___ Python tracker ___

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread STINNER Victor
STINNER Victor added the comment: A different approach for this issue is to document fcntl.fcntl(fd, fcntl.F_FULLFSYNC) in fsync() documentation. -- ___ Python tracker ___

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Ronald Oussoren
Ronald Oussoren added the comment: See for linux' behavior, in particular: linux doesn't guarantee that data gets writting to the disk when you call fsync, only that the data gets pushed to the storage device. This is the s

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-20 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso added the comment: > when one calls fsync, he expects [...] > Fixing this deficiency through Python's exposed fsync [...] I think so, too. http://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html even permits "null implementation"s etc. etc. etc. (I like the la

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-19 Thread Charles-Francois Natali
Charles-Francois Natali added the comment: I know that POSIX makes no guarantee regarding durable writes, but IMHO that's definitely wrong, in the sense that when one calls fsync, he expects the data to be committed to disk and be durable. Fixing this deficiency through Python's exposed fsync

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-19 Thread Ronald Oussoren
Ronald Oussoren added the comment: Traditionally the functions in os are thin wrappers around OS functions, this patch changes that for os.fsync. Two nits wrt. the patch itself: 1) The behaviour on OSX should be documented in the stdlib reference, that currently says that os.fsync will cal

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-19 Thread STINNER Victor
STINNER Victor added the comment: Oh, it remembers a long story around ext3/ext4 and write barrier, with a specific problem in Firefox with SQLite. http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/fsync.2.html " For applications that require tighter guar

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-19 Thread Nadeem Vawda
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue11877] Mac OS X fsync() should really be fcntl(F_FULLFSYNC)

2011-04-19 Thread Steffen Daode Nurpmeso
New submission from Steffen Daode Nurpmeso : Issue 11277 revealed that Mac OS X fsync() does not always and gracefully synchronize file data to physical backing store. To gain real fsync(2) behaviour fcntl(2) must be called with the F_FULLFSYNC flag. -- components: Library (Lib) files: f