Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-22 Thread Robert Haas
On Thu, Nov 21, 2013 at 8:26 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-21 08:22:05 -0500, Robert Haas wrote: On Thu, Nov 21, 2013 at 6:15 AM, Andres Freund and...@2ndquadrant.com wrote: WRT performance: I agree that fixed-width identifiers are more performant, that's why

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-22 Thread Andres Freund
On 2013-11-22 14:43:15 -0500, Robert Haas wrote: The patch as proposed puts forward a particular way of doing that, and I think that neither that method *nor any other* should be part of core. Working on something like that, updated the patch state to waiting on author. Thanks, Andres Freund

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-21 Thread Andres Freund
Hi, On 2013-11-19 18:49:27 -0500, Steve Singer wrote: But for that the receiving side needs to know up to where changes have been applied. One relatively easy solution for that is that the receiving side does something like: UPDATE replication_progress SET lsn = '0/1600' WHERE source_id

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-21 Thread Andres Freund
On 2013-11-19 14:16:04 +, Greg Stark wrote: On Thu, Nov 14, 2013 at 5:26 PM, Andres Freund and...@2ndquadrant.comwrote: But for that the receiving side needs to know up to where changes have been applied. One relatively easy solution for that is that the receiving side does something

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-21 Thread Andres Freund
On 2013-11-20 15:05:17 -0500, Robert Haas wrote: That's what I had suggested to some people originally and the response was, well, somewhat unenthusiastic. It's not that easy to assign them in a meaningful automated manner. How do you automatically assign a pg cluster an id?

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-21 Thread Peter Eisentraut
On 11/21/13, 6:15 AM, Andres Freund wrote: On 2013-11-20 15:05:17 -0500, Robert Haas wrote: That's what I had suggested to some people originally and the response was, well, somewhat unenthusiastic. It's not that easy to assign them in a meaningful automated manner. How do you automatically

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-21 Thread Robert Haas
On Thu, Nov 21, 2013 at 6:15 AM, Andres Freund and...@2ndquadrant.com wrote: WRT performance: I agree that fixed-width identifiers are more performant, that's why I went for them, but I am not sure it's that important. The performance sensitive parts should all be done using the internal

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-21 Thread Andres Freund
On 2013-11-21 08:22:05 -0500, Robert Haas wrote: On Thu, Nov 21, 2013 at 6:15 AM, Andres Freund and...@2ndquadrant.com wrote: WRT performance: I agree that fixed-width identifiers are more performant, that's why I went for them, but I am not sure it's that important. The performance

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-20 Thread Robert Haas
On Tue, Nov 19, 2013 at 1:20 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-19 12:47:29 -0500, Robert Haas wrote: On Tue, Nov 19, 2013 at 11:57 AM, Andres Freund and...@2ndquadrant.com wrote: Agreed. As an alternative we could just have a single - probably longer than

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-19 Thread Robert Haas
On Thu, Nov 14, 2013 at 12:26 PM, Andres Freund and...@2ndquadrant.com wrote: As you know, the reason we are working changeset extraction is that we want to build logical unidirection and bidirectional replication ontop. To use changeset extraction effectively, I think one set of related

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-19 Thread Greg Stark
On Thu, Nov 14, 2013 at 5:26 PM, Andres Freund and...@2ndquadrant.comwrote: But for that the receiving side needs to know up to where changes have been applied. One relatively easy solution for that is that the receiving side does something like: UPDATE replication_progress SET lsn =

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-19 Thread Andres Freund
Hi, On 2013-11-19 07:40:30 -0500, Robert Haas wrote: This way, after a crash, restart, disconnect the replay process can look into shared memory and check how far it has already replayed and restart seamlessly. With minimal effort. It would be much less invasive for the replication apply

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-19 Thread Robert Haas
On Tue, Nov 19, 2013 at 11:57 AM, Andres Freund and...@2ndquadrant.com wrote: Agreed. As an alternative we could just have a single - probably longer than NAMEDATALEN - string to identify replication progress and rely on the users of the facility to build the identifier automatically

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-19 Thread Andres Freund
On 2013-11-19 12:47:29 -0500, Robert Haas wrote: On Tue, Nov 19, 2013 at 11:57 AM, Andres Freund and...@2ndquadrant.com wrote: Agreed. As an alternative we could just have a single - probably longer than NAMEDATALEN - string to identify replication progress and rely on the users of the

Re: [HACKERS] Replication Node Identifiers and crashsafe Apply Progress

2013-11-19 Thread Steve Singer
On 11/14/2013 12:26 PM, Andres Freund wrote: Hello, As you know, the reason we are working changeset extraction is that we want to build logical unidirection and bidirectional replication ontop. To use changeset extraction effectively, I think one set of related features ontop is very useful: