Re: [PERFORM] [HACKERS] fsync method checking

2004-03-26 Thread markw
On 26 Mar, Bruce Momjian wrote: [EMAIL PROTECTED] wrote: On 26 Mar, Manfred Spraul wrote: [EMAIL PROTECTED] wrote: Compare file sync methods with one 8k write: (o_dsync unavailable) open o_sync, write 6.270724 write, fdatasync13.275225

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-26 Thread Bruce Momjian
[EMAIL PROTECTED] wrote: On 26 Mar, Manfred Spraul wrote: [EMAIL PROTECTED] wrote: Compare file sync methods with one 8k write: (o_dsync unavailable) open o_sync, write 6.270724 write, fdatasync13.275225 write, fsync, 13.359847

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-26 Thread markw
On 26 Mar, Manfred Spraul wrote: [EMAIL PROTECTED] wrote: Compare file sync methods with one 8k write: (o_dsync unavailable) open o_sync, write 6.270724 write, fdatasync13.275225 write, fsync, 13.359847 Odd. Which filesystem, which

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-26 Thread Steve Atkins
On Fri, Mar 26, 2004 at 07:25:53AM +0100, Manfred Spraul wrote: Compare file sync methods with one 8k write: (o_dsync unavailable) open o_sync, write 6.270724 write, fdatasync13.275225 write, fsync, 13.359847 Odd. Which filesystem,

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-25 Thread markw
On 25 Mar, Manfred Spraul wrote: Tom Lane wrote: [EMAIL PROTECTED] writes: I could certainly do some testing if you want to see how DBT-2 does. Just tell me what to do. ;) Just do some runs that are identical except for the wal_sync_method setting. Note that this should not have any

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-25 Thread Bruce Momjian
[EMAIL PROTECTED] wrote: I've made a test run that compares fsync and fdatasync: The performance was identical: - with fdatasync: http://khack.osdl.org/stp/290607/ - with fsync: http://khack.osdl.org/stp/290483/ I don't understand why. Mark - is there a battery backed write

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-25 Thread Josh Berkus
Bruce, We don't actually extend the WAL file during writes (preallocated), and the access/modification timestamp is only in seconds, so I wonder of the OS only updates the inode once a second. What else would change in the inode more frequently than once a second? What about really big

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-25 Thread markw
On 22 Mar, Tom Lane wrote: [EMAIL PROTECTED] writes: I could certainly do some testing if you want to see how DBT-2 does. Just tell me what to do. ;) Just do some runs that are identical except for the wal_sync_method setting. Note that this should not have any impact on SELECT

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-25 Thread Manfred Spraul
[EMAIL PROTECTED] wrote: Compare file sync methods with one 8k write: (o_dsync unavailable) open o_sync, write 6.270724 write, fdatasync13.275225 write, fsync, 13.359847 Odd. Which filesystem, which kernel? It seems fdatasync is broken and

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-24 Thread Manfred Spraul
Tom Lane wrote: [EMAIL PROTECTED] writes: I could certainly do some testing if you want to see how DBT-2 does. Just tell me what to do. ;) Just do some runs that are identical except for the wal_sync_method setting. Note that this should not have any impact on SELECT performance, only

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-22 Thread markw
On 18 Mar, Tom Lane wrote: Josh Berkus [EMAIL PROTECTED] writes: 1) This is an OSS project. Why not just recruit a bunch of people on PERFORMANCE and GENERAL to test the 4 different synch methods using real databases? No test like reality, I say I agree --- that is likely to

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-22 Thread Tom Lane
[EMAIL PROTECTED] writes: I could certainly do some testing if you want to see how DBT-2 does. Just tell me what to do. ;) Just do some runs that are identical except for the wal_sync_method setting. Note that this should not have any impact on SELECT performance, only insert/update/delete

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-22 Thread Bruce Momjian
[EMAIL PROTECTED] wrote: On 18 Mar, Tom Lane wrote: Josh Berkus [EMAIL PROTECTED] writes: 1) This is an OSS project. Why not just recruit a bunch of people on PERFORMANCE and GENERAL to test the 4 different synch methods using real databases? No test like reality, I say I

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-19 Thread Kevin Brown
I wrote: Note, too, that the preferred method isn't likely to depend just on the operating system, it's likely to depend also on the filesystem type being used. Linux provides quite a few of them: ext2, ext3, jfs, xfs, and reiserfs, and that's just off the top of my head. I imagine the

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-18 Thread Josh Berkus
Tom, Bruce, My previous point about checking different fsync spacings corresponds to different assumptions about average transaction size. I think a useful tool for determining wal_sync_method has got to be able to reflect that range of possibilities. Questions: 1) This is an OSS project.

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-18 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: 1) This is an OSS project. Why not just recruit a bunch of people on PERFORMANCE and GENERAL to test the 4 different synch methods using real databases? No test like reality, I say I agree --- that is likely to yield *far* more useful results

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-18 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Well, I wrote the program to allow testing. I don't see a complex test as being that much better than simple one. We don't need accurate numbers. We just need to know if fsync or O_SYNC is faster. Faster than what? The thing everyone is trying to

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-18 Thread Kevin Brown
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Well, I wrote the program to allow testing. I don't see a complex test as being that much better than simple one. We don't need accurate numbers. We just need to know if fsync or O_SYNC is faster. Faster than what? The thing

Re: [PERFORM] [HACKERS] fsync method checking

2004-03-18 Thread Bruce Momjian
Tom Lane wrote: It really just shows whether the fsync fater the close has similar timing to the one before the close. That was the best way I could think to test it. Sure, but where's the separate process part? What this seems to test is whether a single process can sync its own