pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On Wed, Dec 29, 2021 at 8:06 PM Bharath Rupireddy wrote: > > On Wed, Dec 29, 2021 at 7:27 PM Stephen Frost wrote: > > > On Thu, Dec 23, 2021, at 9:58 AM, Bharath Rupireddy wrote: > > > > pg_archivecleanup currently takes a WAL file name as input to delete > > > > the WAL files prior to it [1]. As

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

Greetings, * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > Thanks for the comments. Here's a new tool called pg_walcleaner which > basically deletes (optionally archiving before deletion) the unneeded > WAL files. > > Please provide your thoughts and review the patches. Alr

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On Mon, Apr 18, 2022 at 7:41 PM Stephen Frost wrote: > > Greetings, > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > Thanks for the comments. Here's a new tool called pg_walcleaner which > > basically deletes (optionally archiving before deletion) the unneeded > > WAL fi

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

Greeting, * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > On Mon, Apr 18, 2022 at 7:41 PM Stephen Frost wrote: > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > > Thanks for the comments. Here's a new tool called pg_walcleaner which > > > basically

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On Thu, Jun 30, 2022 at 5:33 PM Peter Eisentraut wrote: > > On 25.04.22 20:39, Stephen Frost wrote: > > All of which isn't an issue if we don't have an external tool trying to > > do this and instead have the server doing it as the server knows its > > internal status, that the archive command has

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On Mon, Apr 18, 2022 at 8:48 PM Stephen Frost wrote: > > Greeting, > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > On Mon, Apr 18, 2022 at 7:41 PM Stephen Frost wrote: > > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > > > Thanks for the comm

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

Greetings, * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > On Mon, Apr 18, 2022 at 8:48 PM Stephen Frost wrote: > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > > On Mon, Apr 18, 2022 at 7:41 PM Stephen Frost wrote: > > > > * Bharath Rupireddy (bh

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On Tue, Apr 26, 2022 at 12:09 AM Stephen Frost wrote: > > I was thinking more specifically along the lines of "if there's > X GB > of WAL that hasn't been archived, give up on archiving anything new" > (which is how the pgbackrest option works). IMO, this option is dangerous because the users mig

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

Greetings, * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > On Tue, Apr 26, 2022 at 12:09 AM Stephen Frost wrote: > > I was thinking more specifically along the lines of "if there's > X GB > > of WAL that hasn't been archived, give up on archiving anything new" > > (which is

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On 5/3/22 17:17, Stephen Frost wrote: * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: The pg_walcleaner tool isn't intrusive in the sense that it doesn't delete the WAL files that are required for the server to come up (as it checks for the checkpoint redo WAL file), apart f

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

On 25.04.22 20:39, Stephen Frost wrote: All of which isn't an issue if we don't have an external tool trying to do this and instead have the server doing it as the server knows its internal status, that the archive command has been failing long enough to pass the configuration threshold, and that