Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Tom Lane
Greg Stark writes: > So I'm thinking of something like this. > Ignore ESDIR when opening a directory (and return immediately) > and ignore EBADF when trying to fsync a directory. Seems reasonable, but get rid of the comment "However we can't do this just yet, it has portability issues"; and you'v

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Greg Stark
On Tue, Feb 23, 2010 at 3:38 PM, Tom Lane wrote: >> The plan I was thinking of was to pass a flag indicating if it's a >> directory to fsync_fname() and open it RD_ONLY if it's a directory and >> RDRW if it's a file. Then ignore any error returns (or at least EBADF >> and EINVAL) iff the flag indi

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-23 Thread Tom Lane
I wrote: > BTW, I notice that after allegedly fixing things, we are now seeing > fsync failures during CREATE DATABASE in the installcheck phase of > buildfarm runs on (apparently) all the Windows critters, plus a > couple of other platforms too. This mystifies me. I could believe > that there wa

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-23 Thread Tom Lane
Greg Stark writes: > I don't like using configure tests for this because I fear someone > could compile Postgres on a system with one set of behaviour and then > switch to a different kernel version with a different set of > behaviour. In the worst case it could be filesystem dependent whether > y

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-23 Thread Tom Lane
I wrote: > Any theories about what is happening? Hah --- the AIX failures, at least, are explained at http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/fsync.htm which says Error Codes The fsync or fsync_range subroutine is unsuccessful if on

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-23 Thread Greg Stark
On Tue, Feb 23, 2010 at 5:37 AM, Tom Lane wrote: > So the problem is that fsync_fname is trying to fsync a file it's opened > O_RDONLY.  I don't know whether Windows is similarly picky, but we'll > soon find out. > Argh, now I feel silly. I had actually found that in my searches after the first b

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-22 Thread Andres Freund
Hi Tom, On Tuesday 23 February 2010 06:37:26 Tom Lane wrote: > I wrote: > > Any theories about what is happening? > Now, this doesn't mean that all is fine and dandy. I believe that a > majority of Unixen will reject attempts to open directories for writing, > so this solution puts us even furth

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-22 Thread Tom Lane
BTW, I notice that after allegedly fixing things, we are now seeing fsync failures during CREATE DATABASE in the installcheck phase of buildfarm runs on (apparently) all the Windows critters, plus a couple of other platforms too. This mystifies me. I could believe that there was something still w

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-22 Thread Tom Lane
Greg Stark writes: > On Mon, Feb 22, 2010 at 2:54 AM, Tom Lane wrote: >> I also think it should scan the todir not the fromdir, just on >> general principles to avoid any possibility of race conditions. > I had concluded that scanning the original directory was odd but > better because it served