Re: [HACKERS] Including replication slot data in base backups

2014-04-14 Thread Michael Paquier
On Tue, Apr 15, 2014 at 1:55 AM, Robert Haas wrote: > On Mon, Apr 14, 2014 at 9:26 AM, Fujii Masao wrote: >> This makes me think that it's safer to just remove replication slot files >> at the beginning of the recovery when both backup_label and recovery.conf >> exist. > > Well, we could do that,

Re: [HACKERS] Including replication slot data in base backups

2014-04-14 Thread Robert Haas
On Mon, Apr 14, 2014 at 9:26 AM, Fujii Masao wrote: > This makes me think that it's safer to just remove replication slot files > at the beginning of the recovery when both backup_label and recovery.conf > exist. Well, we could do that, but that would preempt anyone who *does* want to keep those

Re: [HACKERS] Including replication slot data in base backups

2014-04-14 Thread Fujii Masao
On Thu, Apr 10, 2014 at 12:36 AM, Robert Haas wrote: > On Tue, Apr 8, 2014 at 3:08 AM, Michael Paquier > wrote: >> On Tue, Apr 8, 2014 at 1:18 AM, Robert Haas wrote: >>> Not sure if this is exactly the right way to do it, but I agree that >>> something along those lines is a good idea. I also t

Re: [HACKERS] Including replication slot data in base backups

2014-04-13 Thread Michael Paquier
On Thu, Apr 10, 2014 at 12:36 AM, Robert Haas wrote: > On Tue, Apr 8, 2014 at 3:08 AM, Michael Paquier > wrote: >> On Tue, Apr 8, 2014 at 1:18 AM, Robert Haas wrote: >>> Not sure if this is exactly the right way to do it, but I agree that >>> something along those lines is a good idea. I also t

Re: [HACKERS] Including replication slot data in base backups

2014-04-09 Thread Robert Haas
On Tue, Apr 8, 2014 at 3:08 AM, Michael Paquier wrote: > On Tue, Apr 8, 2014 at 1:18 AM, Robert Haas wrote: >> Not sure if this is exactly the right way to do it, but I agree that >> something along those lines is a good idea. I also think, maybe even >> importantly, that we should probably docu

Re: [HACKERS] Including replication slot data in base backups

2014-04-08 Thread Michael Paquier
On Tue, Apr 8, 2014 at 1:18 AM, Robert Haas wrote: > Not sure if this is exactly the right way to do it, but I agree that > something along those lines is a good idea. I also think, maybe even > importantly, that we should probably document that people using > file-copy based hot backups should s

Re: [HACKERS] Including replication slot data in base backups

2014-04-07 Thread Fabrízio de Royes Mello
Em segunda-feira, 7 de abril de 2014, Robert Haas escreveu: > On Mon, Apr 7, 2014 at 3:04 AM, Michael Paquier > > wrote: > > On Fri, Apr 4, 2014 at 9:57 PM, Michael Paquier > > > wrote: > >> For 9.4, clearly yes, this would change the semantic of recovery and > >> this is not something wise to do

Re: [HACKERS] Including replication slot data in base backups

2014-04-07 Thread Robert Haas
On Mon, Apr 7, 2014 at 3:04 AM, Michael Paquier wrote: > On Fri, Apr 4, 2014 at 9:57 PM, Michael Paquier > wrote: >> For 9.4, clearly yes, this would change the semantic of recovery and >> this is not something wise to do at the end of a development cycle. >> For 9.5 though, this is a different s

Re: [HACKERS] Including replication slot data in base backups

2014-04-07 Thread Michael Paquier
On Fri, Apr 4, 2014 at 9:57 PM, Michael Paquier wrote: > For 9.4, clearly yes, this would change the semantic of recovery and > this is not something wise to do at the end of a development cycle. > For 9.5 though, this is a different story. It clearly depends on if > this is though as useful enoug

Re: [HACKERS] Including replication slot data in base backups

2014-04-04 Thread Michael Paquier
On Fri, Apr 4, 2014 at 9:44 PM, Andres Freund wrote: >> I imagine >> that if we want to guarantee the correctness of a replication slot we >> would need to fetch from archives the necessary WAL files needed for >> it when a node is in recovery, which is not something that this patch >> does... > >

Re: [HACKERS] Including replication slot data in base backups

2014-04-04 Thread Andres Freund
On 2014-04-04 21:42:33 +0900, Michael Paquier wrote: > On Wed, Apr 2, 2014 at 10:27 PM, Andres Freund wrote: > > The new master won't necessarily have all the neccessary WAL available, no? > No, they won't have it, and things begin to get bad once a base backup > includes a slot that has a non-nu

Re: [HACKERS] Including replication slot data in base backups

2014-04-04 Thread Michael Paquier
On Wed, Apr 2, 2014 at 10:27 PM, Andres Freund wrote: > The new master won't necessarily have all the neccessary WAL available, no? No, they won't have it, and things begin to get bad once a base backup includes a slot that has a non-null value of restart_lsn. I imagine that if we want to guarante

Re: [HACKERS] Including replication slot data in base backups

2014-04-02 Thread Andres Freund
On 2014-04-02 20:59:03 +0900, Michael Paquier wrote: > On Wed, Apr 2, 2014 at 6:58 PM, Andres Freund wrote: > > On 2014-04-02 09:59:28 +0900, Michael Paquier wrote: > >> On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund > >> wrote: > >> > On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: > >> >>

Re: [HACKERS] Including replication slot data in base backups

2014-04-02 Thread Michael Paquier
On Wed, Apr 2, 2014 at 6:58 PM, Andres Freund wrote: > On 2014-04-02 09:59:28 +0900, Michael Paquier wrote: >> On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund >> wrote: >> > On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: >> >> On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier >> >> wrote: >>

Re: [HACKERS] Including replication slot data in base backups

2014-04-02 Thread Andres Freund
On 2014-04-02 09:59:28 +0900, Michael Paquier wrote: > On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund wrote: > > On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: > >> On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier > >> wrote: > >> > As of now, pg_basebackup creates an empty repository for pg_

Re: [HACKERS] Including replication slot data in base backups

2014-04-02 Thread Bernd Helmle
--On 1. April 2014 11:26:08 -0400 Robert Haas wrote: As a general comment, I think that replication slots, while a great feature, have more than the usual potential for self-inflicted injury. A replication slot prevents the global xmin from advancing (so your tables will bloat) and WAL from

Re: [HACKERS] Including replication slot data in base backups

2014-04-01 Thread Michael Paquier
On Tue, Apr 1, 2014 at 11:59 PM, Andres Freund wrote: > On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: >> On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier >> wrote: >> > As of now, pg_basebackup creates an empty repository for pg_replslot/ >> > in a base backup, forcing the user to recreate s

Re: [HACKERS] Including replication slot data in base backups

2014-04-01 Thread Josh Berkus
On 04/01/2014 05:24 AM, Michael Paquier wrote: > Hi all, > > As of now, pg_basebackup creates an empty repository for pg_replslot/ > in a base backup, forcing the user to recreate slots on other nodes of > the cluster with pg_create_*_replication_slot, or copy pg_replslot > from another node. This

Re: [HACKERS] Including replication slot data in base backups

2014-04-01 Thread Robert Haas
On Tue, Apr 1, 2014 at 10:45 AM, Magnus Hagander wrote: > On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier > wrote: >> >> As of now, pg_basebackup creates an empty repository for pg_replslot/ >> in a base backup, forcing the user to recreate slots on other nodes of >> the cluster with pg_create_*_

Re: [HACKERS] Including replication slot data in base backups

2014-04-01 Thread Andres Freund
On 2014-04-01 16:45:46 +0200, Magnus Hagander wrote: > On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier > wrote: > > > Hi all, > > > > As of now, pg_basebackup creates an empty repository for pg_replslot/ > > in a base backup, forcing the user to recreate slots on other nodes of > > the cluster wit

Re: [HACKERS] Including replication slot data in base backups

2014-04-01 Thread Magnus Hagander
On Tue, Apr 1, 2014 at 2:24 PM, Michael Paquier wrote: > Hi all, > > As of now, pg_basebackup creates an empty repository for pg_replslot/ > in a base backup, forcing the user to recreate slots on other nodes of > the cluster with pg_create_*_replication_slot, or copy pg_replslot > from another no

[HACKERS] Including replication slot data in base backups

2014-04-01 Thread Michael Paquier
Hi all, As of now, pg_basebackup creates an empty repository for pg_replslot/ in a base backup, forcing the user to recreate slots on other nodes of the cluster with pg_create_*_replication_slot, or copy pg_replslot from another node. This is not really user-friendly especially after a failover wh