Re: Allow pg_archivecleanup to remove backup history files

2023-07-19 Thread torikoshia
On 2023-07-19 13:58, Michael Paquier wrote: On Fri, Jun 30, 2023 at 03:48:43PM +0900, Michael Paquier wrote: I have begun cleaning up my board, and applied 0001 for the moment. And a few weeks later.. I have come around this thread and applied 0002 and 0003. The flow of 0002 was straight-for

Re: Allow pg_archivecleanup to remove backup history files

2023-07-18 Thread Michael Paquier
On Fri, Jun 30, 2023 at 03:48:43PM +0900, Michael Paquier wrote: > I have begun cleaning up my board, and applied 0001 for the moment. And a few weeks later.. I have come around this thread and applied 0002 and 0003. The flow of 0002 was straight-forward. My main issue was in 0003, actually, wh

Re: Allow pg_archivecleanup to remove backup history files

2023-06-29 Thread Michael Paquier
On Fri, Jun 23, 2023 at 05:37:09PM +0900, torikoshia wrote: > On 2023-06-22 16:47, Kyotaro Horiguchi wrote: > Thanks for your review! I have begun cleaning up my board, and applied 0001 for the moment. -- Michael signature.asc Description: PGP signature

Re: Allow pg_archivecleanup to remove backup history files

2023-06-23 Thread torikoshia
if (dryrun) + { + /* + * Prints the name of the file to be removed and skips the + * actual removal. The regular printout is so that the + * user can pipe the output into some other program. + */ + printf("%s\n", WALFilePath); + pg_log_debug("file \"%s\" wou

Re: Allow pg_archivecleanup to remove backup history files

2023-06-22 Thread Kyotaro Horiguchi
At Wed, 21 Jun 2023 23:41:33 +0900, torikoshia wrote in > --v10-0002-Preliminary-refactoring-for-a-subsequent-patch.patch > + * Also we skip backup history files when --clean-backup-history > +* is not specified. > +*/ > + if (!IsXLogFileName(walfile) && !IsPartialXLogFileName(wa

Re: Allow pg_archivecleanup to remove backup history files

2023-06-21 Thread torikoshia
*/ + printf("%s\n", WALFilePath); + pg_log_debug("file \"%s\" would be removed", WALFilePath); + continue; + } + + pg_log_debug("removing file \"%s\"", WALFilePath); + + rc = unlink(WALFilePath); + if (rc != 0) + pg_fatal("could

Re: Allow pg_archivecleanup to remove backup history files

2023-06-20 Thread Kyotaro Horiguchi
*/ + printf("%s\n", WALFilePath); + pg_log_debug("file \"%s\" would be removed", WALFilePath); + continue; + } + + pg_log_debug("removing file \"%s\"", WALFilePath); + + rc = unlink(WALFilePath); + if (rc != 0) + pg_fatal("could n

Re: Allow pg_archivecleanup to remove backup history files

2023-06-20 Thread torikoshia
FilePath); + + rc = unlink(WALFilePath); + if (rc != 0) +pg_fatal("could not remove file \"%s\": %m", + WALFilePath); + } } - else - pg_fatal("could not open archive location \"%s\": %m", + + if (errno) + pg_fatal("could not read arc

Re: Allow pg_archivecleanup to remove backup history files

2023-06-18 Thread Michael Paquier
On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote: > Thanks, now I understand what you meant. If I may ask, why is the refactoring of 0003 done after the feature in 0002? Shouldn't the order be reversed? That would make for a cleaner git history. -- Michael signature.asc Description:

Re: Allow pg_archivecleanup to remove backup history files

2023-06-18 Thread torikoshia
tatic struct option long_options[] = { + {"debug", no_argument, NULL, 'd'}, + {"dry-run", no_argument, NULL, 'n'}, + {"strip-extension", required_argument, NULL, 'x'}, + {NULL, 0, NULL, 0} + }; int c; pg_logging_init(argv[0])

Re: Allow pg_archivecleanup to remove backup history files

2023-06-15 Thread Kyotaro Horiguchi
At Fri, 16 Jun 2023 11:22:31 +0900 (JST), Kyotaro Horiguchi wrote in > ASAICS the main section of the "pg_rewind --help" fits within 80 > columns. However, "initdb --help" does output a few lines exceeding > the 80-column limit. Judging by the surrounding lines, I believe we're > still aiming to

Re: Allow pg_archivecleanup to remove backup history files

2023-06-15 Thread Kyotaro Horiguchi
At Thu, 15 Jun 2023 21:38:28 +0900, torikoshia wrote in > On 2023-06-15 15:20, Kyotaro Horiguchi wrote: > Thanks for your review! > > + printf(_(" -x, --strip-extension=EXT strip this extention before > > identifying files fo clean up\n")); > > + printf(_(" -?, --help show this help, then exit\n

Re: Allow pg_archivecleanup to remove backup history files

2023-06-15 Thread torikoshia
long_options[] = { + {"debug", no_argument, NULL, 'd'}, + {"dry-run", no_argument, NULL, 'n'}, + {"strip-extension", required_argument, NULL, 'x'}, + {NULL, 0, NULL, 0} + }; int c; pg_logging_init(argv[0]); @@ -294,7 +300,7 @@ main(int argc

Re: Allow pg_archivecleanup to remove backup history files

2023-06-14 Thread Kyotaro Horiguchi
At Wed, 14 Jun 2023 00:49:39 +0900, torikoshia wrote in > On 2023-06-12 16:33, Michael Paquier wrote: > > On Fri, Jun 09, 2023 at 12:32:15AM +0900, Fujii Masao wrote: > Thanks for reviewing! > > >>printf(_(" -n, --dry-run dry run, show the names of the files that > >>would be removed\n"

Re: Allow pg_archivecleanup to remove backup history files

2023-06-13 Thread torikoshia
'd'}, + {"dry-run", no_argument, NULL, 'n'}, + {"strip-extension", required_argument, NULL, 'x'}, + {NULL, 0, NULL, 0} + }; int c; pg_logging_init(argv[0]); @@ -294,7 +300,7 @@ main(int argc, char **argv) } } - while ((c = getopt(arg

Re: Allow pg_archivecleanup to remove backup history files

2023-06-12 Thread Michael Paquier
On Fri, Jun 09, 2023 at 12:32:15AM +0900, Fujii Masao wrote: > + > + Remove backup history files. > > Isn't it better to document clearly which backup history files to be removed? > For example, "In addition to removing WAL files, remove backup history files > with prefixes logica

Re: Allow pg_archivecleanup to remove backup history files

2023-06-08 Thread Fujii Masao
On 2023/05/31 10:51, torikoshia wrote: Update the patch according to the advice. Thanks for updating the patches! I have small comments regarding 0002 patch. + + Remove backup history files. Isn't it better to document clearly which backup history files to be removed? For e

Re: Allow pg_archivecleanup to remove backup history files

2023-05-30 Thread torikoshia
rgc, char **argv) { + static struct option long_options[] = { + {"debug", no_argument, NULL, 'd'}, + {"dry-run", no_argument, NULL, 'n'}, + {"strip-extension", required_argument, NULL, 'x'}, + {NULL, 0, NULL, 0} + }; int c;

Re: Allow pg_archivecleanup to remove backup history files

2023-05-25 Thread Michael Paquier
On Thu, May 25, 2023 at 11:51:18PM +0900, torikoshia wrote: > Updated patches according to your comment. - ok(!-f "$tempdir/$walfiles[1]", - "$test_name: second older WAL file was cleaned up"); - ok(-f "$tempdir/$walfiles[2]", + ok(!-f "$tempdir/@$walfiles[1]", + "$test_name: sec

Re: Allow pg_archivecleanup to remove backup history files

2023-05-25 Thread Michael Paquier
On Fri, May 26, 2023 at 10:07:48AM +0900, Kyotaro Horiguchi wrote: > + if (!IsXLogFileName(walfile) && > !IsPartialXLogFileName(walfile) && > + (!cleanBackupHistory || > !IsBackupHistoryFileName(walfile))) > + continue; >

Re: Allow pg_archivecleanup to remove backup history files

2023-05-25 Thread Kyotaro Horiguchi
At Thu, 25 May 2023 23:51:18 +0900, torikoshia wrote in > Updated patches according to your comment. + printf(_(" -x --strip-extension=EXTclean up files if they have this extension\n")); Perhaps a comma is needed after "-x". The apparent inconsistency between the long name and the

Re: Allow pg_archivecleanup to remove backup history files

2023-05-25 Thread torikoshia
p [OPTION]... ARCHIVELOCATION %%r'\n" @@ -274,6 +274,12 @@ usage(void) int main(int argc, char **argv) { + static struct option long_options[] = { + {"debug", no_argument, NULL, 'd'}, + {"dry-run", no_argument, NULL, 'n'}, + {"strip-extension&q

Re: Allow pg_archivecleanup to remove backup history files

2023-05-23 Thread Michael Paquier
On Mon, May 22, 2023 at 06:24:49PM +0900, torikoshia wrote: > Thanks for your advice, attached patches. 0001 looks OK, thanks! +Remove files including backup history file. This could be reworded as "Remove backup history files.", I assume. + Note that when oldestkeptwalfile is a ba

Re: Allow pg_archivecleanup to remove backup history files

2023-05-22 Thread torikoshia
ain(int argc, char **argv) } } - while ((c = getopt(argc, argv, "dnx:")) != -1) + while ((c = getopt_long(argc, argv, "dnx:", long_options, NULL)) != -1) { switch (c) { base-commit: ac298d3cb56b015acd40d2e015e07a87d8aff124 -- 2.39.2 From 6017a774691cf52f7f51b817d

Re: Allow pg_archivecleanup to remove backup history files

2023-05-14 Thread Michael Paquier
On Mon, May 15, 2023 at 03:16:46PM +0900, torikoshia wrote: > On 2023-05-15 09:18, Michael Paquier wrote: >> How about plugging in some long options, and use something more >> explicit like --clean-backup-history? > > Agreed. If you begin to implement that, it seems to me that this should be shap

Re: Allow pg_archivecleanup to remove backup history files

2023-05-14 Thread torikoshia
On 2023-05-15 09:18, Michael Paquier wrote: On Fri, May 12, 2023 at 05:53:45PM +0900, torikoshia wrote: On 2023-05-10 17:52, Bharath Rupireddy wrote: I was a little concerned about what to do when deleting both the files ending in .gz and backup history files. Is making it possible to specify bo

Re: Allow pg_archivecleanup to remove backup history files

2023-05-14 Thread Michael Paquier
On Fri, May 12, 2023 at 05:53:45PM +0900, torikoshia wrote: > On 2023-05-10 17:52, Bharath Rupireddy wrote: > I was a little concerned about what to do when deleting both the files > ending in .gz and backup history files. > Is making it possible to specify both "-x .backup" and "-x .gz" the way to

Re: Allow pg_archivecleanup to remove backup history files

2023-05-12 Thread torikoshia
On 2023-05-10 17:52, Bharath Rupireddy wrote: Thanks for your comments! Just curious to know the driving point behind this proposal - is pg_archivecleanup deployed in production that was unable to clean up the history files and there were many such history files left? It will help us know how pg

Re: Allow pg_archivecleanup to remove backup history files

2023-05-10 Thread Bharath Rupireddy
On Tue, May 9, 2023 at 7:03 PM torikoshia wrote: > > Attached a patch with documentation and regression tests. Thanks. I think pg_archivecleanup cleaning up history files makes it a complete feature as there's no need to write custom code/scripts over and above what pg_archivecleanup provides. It

Re: Allow pg_archivecleanup to remove backup history files

2023-05-09 Thread torikoshia
RATIONFrom 7d44134e5de930ce04819285a5b7359e370708d4 Mon Sep 17 00:00:00 2001 From: Atsushi Torikoshi Date: Tue, 9 May 2023 21:52:01 +0900 Subject: [PATCH v2] Allow pg_archivecleanup to remove backup history files Add new option -b to remove files including backup history files older than oldestkeptwalfile. --- doc/sr

Re: Allow pg_archivecleanup to remove backup history files

2023-04-25 Thread Michael Paquier
On Tue, Apr 25, 2023 at 05:29:48PM +0900, Kyotaro Horiguchi wrote: > I thought that we have decided not to do that, but I coundn't find any > discussion about it in the ML archive. Anyway, I think it is great > that we have that option. No objections from here to make that optional. It's been ar

Re: Allow pg_archivecleanup to remove backup history files

2023-04-25 Thread Kyotaro Horiguchi
At Tue, 25 Apr 2023 16:38:16 +0900, torikoshia wrote in > Hi, > > Currently pg_archivecleanup doesn't remove backup history files even > when they're older than oldestkeptwalfile. > > Of course the size of backup history files are smaller than WAL files > and they wouldn't consume much disk sp

Allow pg_archivecleanup to remove backup history files

2023-04-25 Thread torikoshia
CORPORATIONFrom ad87224ec5ba6ee13ccf934bf3e5adefb7e67212 Mon Sep 17 00:00:00 2001 From: Atsushi Torikoshi Date: Mon, 24 Apr 2023 23:28:06 +0900 Subject: [PATCH v1] Allow pg_archivecleanup to remove backup history files Add new option -b to remove files including backup history files older than ol