Re: [PERFORM] POSIX file updates

2008-04-17 Thread Greg Smith
On Mon, 31 Mar 2008, James Mansion wrote: I have a question about file writes, particularly on POSIX. In other reading I just came across this informative article on this issue, which amusingly was written the same day you asked about this: http://jeffr-tech.livejournal.com/20707.html -- *

Re: [PERFORM] POSIX file updates

2008-04-02 Thread James Mansion
Greg Smith wrote: You turn on direct I/O differently under Solaris then everywhere else, and nobody has bothered to write the patch (trivial) and OS-specific code to turn it on only when appropriate (slightly tricker) to handle this case. There's not a lot of pressure on PostgreSQL to handle th

Re: [PERFORM] POSIX file updates

2008-04-02 Thread Greg Smith
On Wed, 2 Apr 2008, James Mansion wrote: I'm well aware that there are battery-backed caches that can be detached from controllers and moved. But you'd better make darn sure you move all the drives and plug them in in exactly the right order and make sure they all spin up OK with the replaced

Re: [PERFORM] POSIX file updates

2008-04-02 Thread Greg Smith
On Wed, 2 Apr 2008, James Mansion wrote: But amusingly, PostgreSQL doesn't even support Solaris's direct I/O method right now unless you override the filesystem mounting options, so you end up needing to split it out and hack at that level regardless. Indeed that's a shame. Why doesn't it use

Re: [PERFORM] POSIX file updates

2008-04-02 Thread James Mansion
Andreas Kostyrka wrote: takes over. The thing you worry about is if all data has made it to the replication servers, not if some data might get lost in the hardware cache of a controller. (Actually, talk to your local computer forensics guru, there are a number of way to keep the current to elect

Re: [PERFORM] POSIX file updates

2008-04-02 Thread Andreas Kostyrka
Am Mittwoch, den 02.04.2008, 20:10 +0100 schrieb James Mansion: > It strikes me as odd that fsync_writethrough isn't the most preferred > option where > it is implemented. The postgres approach of *requiring* that there be no > cache > below the OS is problematic, especially since the battery ba

Re: [PERFORM] POSIX file updates

2008-04-02 Thread James Mansion
Greg Smith wrote: "After a write() to a regular file has successfully returned, any successful read() from each byte position in the file that was modified by that write() will return the data that was written by the write()...a similar requirement applies to multiple write operations to the s

Re: [PERFORM] POSIX file updates

2008-03-31 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > Quoting from Lewine's "POSIX Programmer's Guide": > "After a write() to a regular file has successfully returned, any > successful read() from each byte position in the file that was modified by > that write() will return the data that was written by the

Re: [PERFORM] POSIX file updates

2008-03-31 Thread Greg Smith
On Mon, 31 Mar 2008, James Mansion wrote: Is it correct that POSIX requires that the updates to a single file are serialised in the filesystem layer? Quoting from Lewine's "POSIX Programmer's Guide": "After a write() to a regular file has successfully returned, any successful read() from eac

Re: [PERFORM] POSIX file updates

2008-03-31 Thread James Mansion
Mark Mielke wrote: Is there anything in POSIX that seems to suggest this? :-) (i.e. why are you going under the assumption that the answer is yes - did you read something?) Perhaps it was just this: http://kevinclosson.wordpress.com/2007/01/18/yes-direct-io-means-concurrent-writes-oracle-doe

Re: [PERFORM] POSIX file updates

2008-03-31 Thread James Mansion
I don't believe POSIX has any restriction such as you describe - or if it does, and I don't know about it, then most UNIX file systems (if not most file systems on any platform) are not POSIX compliant. I suspect that indeed there are two different issues here in that the file mutex relates

Re: [PERFORM] POSIX file updates

2008-03-31 Thread James Mansion
Mark Mielke wrote: Is there anything in POSIX that seems to suggest this? :-) (i.e. why are you going under the assumption that the answer is yes - did you read something?) It was something somewhere on the Sun web site, relating to tuning Solaris filesystems. Or databases. Or ZFS. :-( Need

Re: [PERFORM] POSIX file updates

2008-03-31 Thread Mark Mielke
James Mansion wrote: (Declaration of interest: I'm researching for a publication on OLTP system design) I have a question about file writes, particularly on POSIX. This arose while considering the extent to which cache memory and command queueing on disk drives can help improve performance. Is

[PERFORM] POSIX file updates

2008-03-31 Thread James Mansion
(Declaration of interest: I'm researching for a publication on OLTP system design) I have a question about file writes, particularly on POSIX. This arose while considering the extent to which cache memory and command queueing on disk drives can help improve performance. Is it correct that POSIX