Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Magnus Hagander
2010/2/15 Greg Stark : > On Mon, Feb 15, 2010 at 11:34 AM, marcin mank wrote: >> LOG:  could not link file "pg_xlog/xlogtemp.2367" to >> "pg_xlog/0001" (initialization of log file 0, >> > > This is not related -- it seems your filesystem doesn't support hard > links. I thought

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Andres Freund
On Monday 15 February 2010 14:50:03 marcin mank wrote: > Yes, the issue with initdb failing is unrelated (and I have no problem > about the fs being unsupported). But fsync still DOES fail on > directories from the mount. > > >> But I would not be that sure that eg. NFS or something like that won`

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread marcin mank
Yes, the issue with initdb failing is unrelated (and I have no problem about the fs being unsupported). But fsync still DOES fail on directories from the mount. >> But I would not be that sure that eg. NFS or something like that won`t >> complain. > It does not. > What if someone mounts a NFS sha

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Andres Freund
On Monday 15 February 2010 12:55:36 Greg Stark wrote: > On Mon, Feb 15, 2010 at 11:50 AM, Andres Freund wrote: > > If I understood him correctly marcin seems to mount a windows share on > > linux via some vbox-proprietary pseudo filesystem. That wont get > > detected and thus no junctions will be

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 11:50 AM, Andres Freund wrote: > If I understood him correctly marcin seems to mount a windows share on linux > via some vbox-proprietary pseudo filesystem. That wont get detected and thus > no junctions will be used... (I have doubts you even can create them via > vboxfs

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Andres Freund
On Monday 15 February 2010 12:45:39 Greg Stark wrote: > On Mon, Feb 15, 2010 at 11:34 AM, marcin mank wrote: > > LOG: could not link file "pg_xlog/xlogtemp.2367" to > > "pg_xlog/0001" (initialization of log file 0, > > This is not related -- it seems your filesystem doesn't s

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Andres Freund
On Monday 15 February 2010 12:34:44 marcin mank wrote: > On Mon, Feb 15, 2010 at 11:02 AM, Andres Freund wrote: > > Hi Marcin, > > > > Sounds rather unlikely to me. Its likely handled at an upper layer (vfs > > in linux' case) and only overloaded when an optimized implementation is > > available.

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 11:34 AM, marcin mank wrote: > LOG:  could not link file "pg_xlog/xlogtemp.2367" to > "pg_xlog/0001" (initialization of log file 0, > This is not related -- it seems your filesystem doesn't support hard links. I thought we used "junctions" on versions o

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread marcin mank
On Mon, Feb 15, 2010 at 11:02 AM, Andres Freund wrote: > Hi Marcin, > > Sounds rather unlikely to me. Its likely handled at an upper layer (vfs in > linux' case) and only overloaded when an optimized implementation is > available. > Which os do you see implementing that only on a part of the fil

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Greg Stark
On Mon, Feb 15, 2010 at 10:02 AM, Andres Freund wrote: > Hi Marcin, > > Sounds rather unlikely to me. Its likely handled at an upper layer (vfs in > linux' case) and only overloaded when an optimized implementation is > available. > Which os do you see implementing that only on a part of the fil

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Andres Freund
Hi Marcin, Sounds rather unlikely to me. Its likely handled at an upper layer (vfs in linux' case) and only overloaded when an optimized implementation is available. Which os do you see implementing that only on a part of the filesystems? A runtime check would be creating, fsyncing and deleting

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread marcin mank
On Mon, Feb 15, 2010 at 9:36 AM, Andres Freund wrote: > On Monday 15 February 2010 08:13:32 Tom Lane wrote: >> Actually, looking closer, some of the Windows machines started failing >> after the *earlier* patch to add directory fsyncs. > And not only the windows machines. Seems sensible to add a c

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-15 Thread Andres Freund
On Monday 15 February 2010 08:13:32 Tom Lane wrote: > I wrote: > > The buildfarm indicates that this patch has got some serious issues. > > Actually, looking closer, some of the Windows machines started failing > after the *earlier* patch to add directory fsyncs. And not only the windows machines.

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-14 Thread Tom Lane
I wrote: > The buildfarm indicates that this patch has got some serious issues. Actually, looking closer, some of the Windows machines started failing after the *earlier* patch to add directory fsyncs. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-14 Thread Tom Lane
st...@postgresql.org (Greg Stark) writes: > Log Message: > --- > Speed up CREATE DATABASE by deferring the fsyncs until after copying > all the data and using posix_fadvise to nudge the OS into flushing it > earlier. This also hopefully makes CREATE DATABASE avoid spamming the > cache. The