Re: Speed up the removal of WAL files

On Fri, Mar 09, 2018 at 12:50:03AM +, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:mich...@paquier.xyz] >> Hm. durable_xx should remain a sane operation as an isolated call as you >> still get the same problem if a crash happens before flushing the parent... >> Fujii-san idea

Re: Speed up the removal of WAL files

On Wed, Feb 21, 2018 at 4:52 PM, Michael Paquier wrote: > On Wed, Feb 21, 2018 at 07:20:00AM +, Tsunakawa, Takayuki wrote: >> Right. Then I thought of doing the following to avoid making a new >> function only for RemoveNonParentXlogFiles() which is similar to >> RemoveXlogFile(). >> >> *

RE: Speed up the removal of WAL files

From: Michael Paquier [mailto:mich...@paquier.xyz] > Hm. durable_xx should remain a sane operation as an isolated call as you > still get the same problem if a crash happens before flushing the parent... > Fujii-san idea also has value to speed up the end of recovery but this costs > as well in

RE: Speed up the removal of WAL files

From: Michael Paquier [mailto:mich...@paquier.xyz] > On Wed, Mar 07, 2018 at 12:55:43AM +0900, Fujii Masao wrote: > > So, what about, as another approach, making the checkpointer instead > > of the startup process call RemoveNonParentXlogFiles() when > > end-of-recovery checkpoint is executed?

Re: Speed up the removal of WAL files

On Wed, Mar 07, 2018 at 12:55:43AM +0900, Fujii Masao wrote: > So, what about, as another approach, making the checkpointer instead of > the startup process call RemoveNonParentXlogFiles() when end-of-recovery > checkpoint is executed? ISTM that a recovery doesn't need to wait for >

Re: Speed up the removal of WAL files

On Wed, Feb 21, 2018 at 5:27 PM, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:mich...@paquier.xyz] > It seems to me that you would reintroduce partially the problems that >> 1d4a0ab1 has fixed. In short, if a crash happens in the code paths calling

RE: Speed up the removal of WAL files

From: Michael Paquier [mailto:mich...@paquier.xyz] It seems to me that you would reintroduce partially the problems that > 1d4a0ab1 has fixed. In short, if a crash happens in the code paths calling > RemoveXlogFile with durable = false before fsync'ing pg_wal, then a rename > has no guarantee to

Re: Speed up the removal of WAL files

On Wed, Feb 21, 2018 at 07:20:00AM +, Tsunakawa, Takayuki wrote: > Right. Then I thought of doing the following to avoid making a new > function only for RemoveNonParentXlogFiles() which is similar to > RemoveXlogFile(). > > * Add an argument "bool durable" to RemoveXlogFile(). Based on

RE: Speed up the removal of WAL files

From: Fujii Masao [mailto:masao.fu...@gmail.com] > On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki > wrote: > > Yes, I noticed it after submitting the patch and was wondering what to > do. Thinking simply, I think it would be just enough to replace >

RE: Speed up the removal of WAL files

Fujii Masao > Cc: Tsunakawa, Takayuki/綱川 貴之; Kyotaro HORIGUCHI; > pgsql-hack...@postgresql.org > Subject: Re: Speed up the removal of WAL files > > On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao.fu...@gmail.com> wrote: > > On Fri, Nov 17, 2017 at 5:20 PM

Re: Speed up the removal of WAL files

On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki wrote: > From: Kyotaro HORIGUCHI [mailto:horiguchi.kyot...@lab.ntt.co.jp] >> The orinal code recycles some of the to-be-removed files, but the patch >> removes all the victims. This may impact on performance. >

RE: Speed up the removal of WAL files

From: Kyotaro HORIGUCHI [mailto:horiguchi.kyot...@lab.ntt.co.jp] > The orinal code recycles some of the to-be-removed files, but the patch > removes all the victims. This may impact on performance. Yes, I noticed it after submitting the patch and was wondering what to do. Thinking simply, I