Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-29 Thread David Steele
On 9/28/16 9:55 PM, Peter Eisentraut wrote: > On 9/28/16 2:45 AM, Michael Paquier wrote: >> After all that fixed, I have moved the patch to "Ready for Committer". >> Please use the updated patch though. > > Committed after some cosmetic changes. Thank you, Peter! -- -David da...@pgmasters.net

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-28 Thread Peter Eisentraut
On 9/28/16 2:45 AM, Michael Paquier wrote: > After all that fixed, I have moved the patch to "Ready for Committer". > Please use the updated patch though. Committed after some cosmetic changes. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Rem

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-28 Thread David Steele
On 9/28/16 2:45 AM, Michael Paquier wrote: > On Tue, Sep 27, 2016 at 11:27 PM, David Steele wrote: >> On 9/26/16 2:36 AM, Michael Paquier wrote: >> >>> Just a nit: >>> >>> - postmaster.pid >>> + postmaster.pid and postmaster.opts >>> >>> Having one block for ea

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-27 Thread Michael Paquier
On Tue, Sep 27, 2016 at 11:27 PM, David Steele wrote: > On 9/26/16 2:36 AM, Michael Paquier wrote: > >> Just a nit: >> >> - postmaster.pid >> + postmaster.pid and postmaster.opts >> >> Having one block for each file would be better. > > OK, changed. You did no

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-27 Thread David Steele
On 9/26/16 2:36 AM, Michael Paquier wrote: > Just a nit: > > - postmaster.pid > + postmaster.pid and postmaster.opts > > Having one block for each file would be better. OK, changed. > +const char *excludeFile[] = > excludeFiles[]? > > +# Move pg_replslot out

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-27 Thread David Steele
On 9/26/16 2:36 AM, Michael Paquier wrote: > > Just a nit: > > - postmaster.pid > + postmaster.pid and postmaster.opts > > Having one block for each file would be better. I grouped these because they are related and because there are now other bullets where mu

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-25 Thread Michael Paquier
On Sat, Sep 24, 2016 at 3:26 AM, David Steele wrote: > On 9/12/16 2:50 PM, Peter Eisentraut wrote: >> The comments on excludeDirContents are somewhat imprecise. For >> example, none of those directories are actually removed or recreated >> on startup, only the contents are. > > Fixed. > >> The co

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-23 Thread David Steele
On 9/12/16 2:50 PM, Peter Eisentraut wrote: > The comments on excludeDirContents are somewhat imprecise. For > example, none of those directories are actually removed or recreated > on startup, only the contents are. Fixed. > The comment for pg_replslot is incorrect. I think you can copy > repl

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-12 Thread Michael Paquier
On Tue, Sep 13, 2016 at 3:50 AM, Peter Eisentraut wrote: > Add some tests. At least test that one entry from the directory list > and one entry from the files list is not contained in the backup > result. Testing the symlink handling would also be good. (The > pg_basebackup tests claim that Win

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-12 Thread Peter Eisentraut
The comments on excludeDirContents are somewhat imprecise. For example, none of those directories are actually removed or recreated on startup, only the contents are. The comment for pg_replslot is incorrect. I think you can copy replication slots just fine, but you usually don't want to. What

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-10 Thread Michael Paquier
On Fri, Sep 9, 2016 at 10:18 PM, David Steele wrote: > On 9/6/16 10:25 PM, Michael Paquier wrote: >> >> >> On Wed, Sep 7, 2016 at 12:16 AM, David Steele wrote: >> >>> Attached is a new patch that adds sgml documentation. I can expand on >>> each >>> directory individually if you think that's nec

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-09 Thread David Steele
On 9/6/16 10:25 PM, Michael Paquier wrote: > On Wed, Sep 7, 2016 at 12:16 AM, David Steele wrote: Attached is a new patch that adds sgml documentation. I can expand on each directory individually if you think that's necessary, but thought it was better to lump them into a few categories. +

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-07 Thread David Steele
On 9/7/16 8:21 AM, David Steele wrote: > On 9/6/16 10:25 PM, Michael Paquier wrote: >> I find that ugly. I'd rather use an array with undefined size for the >> fixed elements finishing by NULL, remove EXCLUDE_DIR_MAX and >> EXCLUDE_DIR_STAT_TMP and use a small routine to do the work done on >> _tar

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-07 Thread David Steele
On 9/6/16 10:25 PM, Michael Paquier wrote: > On Wed, Sep 7, 2016 at 12:16 AM, David Steele wrote: >> Attached is a new patch that adds sgml documentation. I can expand on each >> directory individually if you think that's necessary, but thought it was >> better to lump them into a few categories.

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-06 Thread Michael Paquier
On Wed, Sep 7, 2016 at 12:16 AM, David Steele wrote: > On 9/1/16 9:53 AM, Peter Eisentraut wrote: >> >> On 8/15/16 3:39 PM, David Steele wrote: >>> >>> That patch got me thinking about what else could be excluded and after >>> some investigation I found the following: pg_notify, pg_serial, >>> pg_

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-06 Thread David Steele
On 9/1/16 9:53 AM, Peter Eisentraut wrote: On 8/15/16 3:39 PM, David Steele wrote: That patch got me thinking about what else could be excluded and after some investigation I found the following: pg_notify, pg_serial, pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, or rebu

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-01 Thread Peter Eisentraut
On 8/15/16 3:39 PM, David Steele wrote: > That patch got me thinking about what else could be excluded and after > some investigation I found the following: pg_notify, pg_serial, > pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, > or rebuilt on server start. > > The attached

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-18 Thread David Steele
On 8/17/16 7:56 PM, Michael Paquier wrote: > On Thu, Aug 18, 2016 at 1:35 AM, Alvaro Herrera > wrote: >> I don't remember how pg_snapshot works, but it's probably fine >> to start with an empty subdir (is it possible to export a snapshot from >> a prepared transaction?) > > From xact.c: > /*

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread Michael Paquier
On Thu, Aug 18, 2016 at 1:35 AM, Alvaro Herrera wrote: > I don't remember how pg_snapshot works, but it's probably fine > to start with an empty subdir (is it possible to export a snapshot from > a prepared transaction?) >From xact.c: /* * Likewise, don't allow PREPARE after pg_export_sn

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread David Steele
On 8/17/16 2:53 PM, Robert Haas wrote: > On Wed, Aug 17, 2016 at 2:50 PM, David Steele wrote: >> Hi Robert, >> >> On 8/17/16 11:27 AM, Robert Haas wrote: >>> On Mon, Aug 15, 2016 at 3:39 PM, David Steele wrote: Recently a hacker proposed a patch to add pg_dynshmem to the list of directo

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread Kevin Grittner
On Wed, Aug 17, 2016 at 1:50 PM, David Steele wrote: >>> That patch got me thinking about what else could be excluded and after >>> some investigation I found the following: pg_notify, pg_serial, >>> pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, >>> or rebuilt on server s

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread Robert Haas
On Wed, Aug 17, 2016 at 2:50 PM, David Steele wrote: > Hi Robert, > > On 8/17/16 11:27 AM, Robert Haas wrote: >> On Mon, Aug 15, 2016 at 3:39 PM, David Steele wrote: >>> Recently a hacker proposed a patch to add pg_dynshmem to the list of >>> directories whose contents are excluded in pg_baseback

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread David Steele
Hi Robert, On 8/17/16 11:27 AM, Robert Haas wrote: > On Mon, Aug 15, 2016 at 3:39 PM, David Steele wrote: >> Recently a hacker proposed a patch to add pg_dynshmem to the list of >> directories whose contents are excluded in pg_basebackup. I wasn't able >> to find the original email despite sever

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread Alvaro Herrera
Robert Haas wrote: > Eh ... I doubt very much that it's safe to blow away the entire > contents of an SLRU between shutdown and startup, even if the data is > technically transient data that won't be needed again after the system > is reset. Hmm. At least async.c (pg_notify) deletes the whole di

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread Robert Haas
On Mon, Aug 15, 2016 at 3:39 PM, David Steele wrote: > Recently a hacker proposed a patch to add pg_dynshmem to the list of > directories whose contents are excluded in pg_basebackup. I wasn't able > to find the original email despite several attempts. > > That patch got me thinking about what el

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-15 Thread David Steele
On 8/15/16 4:58 PM, Jim Nasby wrote: On 8/15/16 2:39 PM, David Steele wrote: That patch got me thinking about what else could be excluded and after some investigation I found the following: pg_notify, pg_serial, pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, or rebuilt on

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-15 Thread Jim Nasby
On 8/15/16 2:39 PM, David Steele wrote: That patch got me thinking about what else could be excluded and after some investigation I found the following: pg_notify, pg_serial, pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, or rebuilt on server start. If someone wanted to

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-15 Thread Stephen Frost
David, * David Steele (da...@pgmasters.net) wrote: > Recently a hacker proposed a patch to add pg_dynshmem to the list of > directories whose contents are excluded in pg_basebackup. I wasn't able > to find the original email despite several attempts. That would be here: b4e94836-786b-6020-e1b3-