Re: [HACKERS] WAL and commit_delay

2001-02-19 Thread Tom Lane
Matthew Kirkwood <[EMAIL PROTECTED]> writes: > BTW, I have attached two patches which streamline log initialisation > a little. The first (xlog-sendfile.diff) adds support for Linux's > sendfile system call. FreeBSD and HP/UX have sendfile() too, but the > prototype is different. If it's intere

Re: [HACKERS] WAL and commit_delay

2001-02-19 Thread Matthew Kirkwood
On Mon, 19 Feb 2001, Matthew Kirkwood wrote: > BTW, I have attached two patches which streamline log initialisation > a little. The first (xlog-sendfile.diff) adds support for Linux's > sendfile system call. Whoops, don't use this. It looks like Linux won't sendfile() from /dev/zero. I'll end

Re: [HACKERS] WAL and commit_delay

2001-02-19 Thread Matthew Kirkwood
On Sun, 18 Feb 2001, Tom Lane wrote: > I think that there may be a performance advantage to pre-filling the > logfile even so, assuming that file allocation info is stored in a > Berkeley/McKusik-like fashion (note: I have no idea what ext2 or > reiserfs actually do). ext2 is a lot like [UF]FS.

Re: [HACKERS] WAL and commit_delay

2001-02-18 Thread Tom Lane
Jerome Vouillon <[EMAIL PROTECTED]> writes: > Actually, there is also a performance reason. Indeed, fdatasync would > not perform any better than fsync if the log file was not > preallocated: the file length would change each time a record is > appended, and therefore the inode would have to be up

Re: [HACKERS] WAL and commit_delay

2001-02-18 Thread Jerome Vouillon
Tom Lane <[EMAIL PROTECTED]> writes: > The implication is that the only thing you can lose after fdatasync is > the highly-inessential file mod time. However, I have been told that > on some implementations, fdatasync only flushes data blocks, and never > writes the inode or indirect blocks. Th

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Dominic J. Eidson
On Sat, 17 Feb 2001, Tom Lane wrote: > Another thing I am wondering about is why we're not using fdatasync(), > where available, instead of fsync(). The whole point of preallocating > the WAL files is to make fdatasync safe, no? Linux/x86 fdatasync(2) manpage: BUGS Currently (Linux 2.0.

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Nathan Myers
On Sat, Feb 17, 2001 at 03:45:30PM -0500, Bruce Momjian wrote: > > Right now the WAL preallocation code (XLogFileInit) is not good enough > > because it does lseek to the 16MB position and then writes 1 byte there. > > On an implementation that supports holes in files (which is most Unixen) > > th

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Tom Lane
Larry Rosenman <[EMAIL PROTECTED]> writes: > I've written swap files and such with: > dd if=/dev/zero of=SWAPFILE bs=512 count=204800 > and all the blocks are allocated. I've also confirmed that writing zeroes is sufficient on HPUX (du shows that the correct amount of space is allocated, unlike

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Bruce Momjian
> * Bruce Momjian <[EMAIL PROTECTED]> [010217 14:46]: > > > Right now the WAL preallocation code (XLogFileInit) is not good enough > > > because it does lseek to the 16MB position and then writes 1 byte there. > > > On an implementation that supports holes in files (which is most Unixen) > > > tha

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Larry Rosenman
* Bruce Momjian <[EMAIL PROTECTED]> [010217 14:50]: > > * Bruce Momjian <[EMAIL PROTECTED]> [010217 14:46]: > > > > Right now the WAL preallocation code (XLogFileInit) is not good enough > > > > because it does lseek to the 16MB position and then writes 1 byte there. > > > > On an implementation t

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Larry Rosenman
* Bruce Momjian <[EMAIL PROTECTED]> [010217 14:46]: > > Right now the WAL preallocation code (XLogFileInit) is not good enough > > because it does lseek to the 16MB position and then writes 1 byte there. > > On an implementation that supports holes in files (which is most Unixen) > > that doesn't

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Bruce Momjian
> Right now the WAL preallocation code (XLogFileInit) is not good enough > because it does lseek to the 16MB position and then writes 1 byte there. > On an implementation that supports holes in files (which is most Unixen) > that doesn't cause physical allocation of the intervening space. We'd >

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Another thing I am wondering about is why we're not using fdatasync(), > where available, instead of fsync(). The whole point of preallocating > the WAL files is to make fdatasync safe, no? > Don't we have to fsync the inode too? Actually, I was hopin

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Bruce Momjian
> > Another thing I am wondering about is why we're not using fdatasync(), > > where available, instead of fsync(). The whole point of preallocating > > the WAL files is to make fdatasync safe, no? > > This still looks like it'd be a win, by reducing the number of seeks > needed to complete a WA

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Bruce Momjian
> Actually ... take a close look at the code. The delay is done in > xact.c between XLogInsert(commitrecord) and XLogFlush(). As near > as I can tell, both the write() and the fsync() will happen in > XLogFlush(). This means the delay is just plain broken: placed > there, it cannot do anything

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Tom Lane
I wrote: > Actually ... take a close look at the code. The delay is done in > xact.c between XLogInsert(commitrecord) and XLogFlush(). As near > as I can tell, both the write() and the fsync() will happen in > XLogFlush(). This means the delay is just plain broken: placed > there, it cannot do

Re: [HACKERS] WAL and commit_delay

2001-02-17 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > With the delay, it looks like: > time backend 1 backend 2 > - - > 0 write() > 1 sleep() write() > 2 fsync() sleep() > 3 fsync() Actually ... take a close look a

[HACKERS] WAL and commit_delay

2001-02-17 Thread Bruce Momjian
I want to give some background on commit_delay, its initial purpose, and possible options. First, looking at the process that happens during a commit: write() - copy WAL dirty page to kernel disk buffer fsync() - force WAL kernel disk buffer to disk platter fsync() take much lon