Re: Fsync request queue

2018-05-01 Thread Andres Freund
On 2018-05-01 13:43:14 -0400, Robert Haas wrote: > On Tue, May 1, 2018 at 1:41 PM, Andres Freund wrote: > > I unfortunately don't have access to the relevant reports anymore, so > > it's only by memory. What I do remember is that a few I saw > > pg_stat_bgwriter.buffers_backend_fsync values that w

Re: Fsync request queue

2018-05-01 Thread Robert Haas
On Tue, May 1, 2018 at 1:41 PM, Andres Freund wrote: > I unfortunately don't have access to the relevant reports anymore, so > it's only by memory. What I do remember is that a few I saw > pg_stat_bgwriter.buffers_backend_fsync values that we a pretty sizable > fraction of the buffers written by b

Re: Fsync request queue

2018-05-01 Thread Andres Freund
On 2018-05-01 13:21:21 -0400, Robert Haas wrote: > On Mon, Apr 30, 2018 at 7:08 PM, Andres Freund wrote: > >> True, but has anyone ever actually observed a non-zero > >> pg_stat_bgwriter.buffers_backend_fsync in the wild after the > >> compaction queue stuff was added/backpatched? > > > > Yes. >

Re: Fsync request queue

2018-05-01 Thread Robert Haas
On Mon, Apr 30, 2018 at 7:08 PM, Andres Freund wrote: >> True, but has anyone ever actually observed a non-zero >> pg_stat_bgwriter.buffers_backend_fsync in the wild after the >> compaction queue stuff was added/backpatched? > > Yes. Care to elaborate? -- Robert Haas EnterpriseDB: http://www.en

Re: Fsync request queue

2018-04-30 Thread Andres Freund
On 2018-04-30 16:07:48 -0700, Peter Geoghegan wrote: > On Mon, Apr 30, 2018 at 4:03 PM, Andres Freund wrote: > >> Is this a problem in practice, though? I don't remember seeing any reports > >> of the fsync queue filling up, after we got the code to compact it. I don't > >> know if anyone has been

Re: Fsync request queue

2018-04-30 Thread Peter Geoghegan
On Mon, Apr 30, 2018 at 4:03 PM, Andres Freund wrote: >> Is this a problem in practice, though? I don't remember seeing any reports >> of the fsync queue filling up, after we got the code to compact it. I don't >> know if anyone has been looking for that, so that might also explain the >> absence

Re: Fsync request queue

2018-04-30 Thread Andres Freund
On 2018-04-25 09:19:52 +0300, Heikki Linnakangas wrote: > On 24/04/18 21:00, Andres Freund wrote: > > Right now if the queue is full and can't be compacted we end up > > fsync()ing on every single write, rather than once per checkpoint > > afaict. That's a fairly horrible. > > > > For the case tha

Re: Fsync request queue

2018-04-24 Thread Heikki Linnakangas
On 24/04/18 21:00, Andres Freund wrote: Right now if the queue is full and can't be compacted we end up fsync()ing on every single write, rather than once per checkpoint afaict. That's a fairly horrible. For the case that there's no space in the map, I'd suggest to just do 10% or so of the fsync