Re: [HACKERS] logical changeset generation v6

2014-04-24 Thread Magnus Hagander
On Mon, Sep 23, 2013 at 7:03 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Sep 23, 2013 at 9:54 AM, Andres Freund and...@2ndquadrant.com wrote: I still find it wierd/inconsistent to have: * pg_receivexlog * pg_recvlogical binaries, even from the same source directory. Why once

Re: [HACKERS] logical changeset generation v6

2014-04-24 Thread Andres Freund
On 2014-04-24 09:39:21 +0200, Magnus Hagander wrote: I can't find that this discussion actually came to a proper consensus, but I may be missing something. Did we go with pg_recvlogical just because we couldn't decide on a better name, or did we intentionally decide it was the best? I went

Re: [HACKERS] logical changeset generation v6

2014-04-24 Thread Magnus Hagander
On Thu, Apr 24, 2014 at 9:43 AM, Andres Freund and...@2ndquadrant.comwrote: On 2014-04-24 09:39:21 +0200, Magnus Hagander wrote: I can't find that this discussion actually came to a proper consensus, but I may be missing something. Did we go with pg_recvlogical just because we couldn't

Re: [HACKERS] logical changeset generation v6

2014-04-24 Thread Andres Freund
On 2014-04-24 09:46:07 +0200, Magnus Hagander wrote: On Thu, Apr 24, 2014 at 9:43 AM, Andres Freund and...@2ndquadrant.comwrote: On 2014-04-24 09:39:21 +0200, Magnus Hagander wrote: I can't find that this discussion actually came to a proper consensus, but I may be missing something.

Re: [HACKERS] logical changeset generation v6

2013-09-28 Thread Steve Singer
On 09/27/2013 05:18 PM, Andres Freund wrote: Hi Steve, On 2013-09-27 17:06:59 -0400, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (set add

Re: [HACKERS] logical changeset generation v6

2013-09-27 Thread Steve Singer
On 09/26/2013 02:47 PM, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (set add table, set add sequence). This is before the SUBSCRIBE SET

Re: [HACKERS] logical changeset generation v6

2013-09-27 Thread Andres Freund
Hi Steve, On 2013-09-27 17:06:59 -0400, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (set add table, set add sequence). This is before the

Re: [HACKERS] logical changeset generation v6

2013-09-26 Thread Steve Singer
On 09/25/2013 01:20 PM, Steve Singer wrote: On 09/25/2013 11:08 AM, Andres Freund wrote: On 2013-09-25 11:01:44 -0400, Steve Singer wrote: On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit Rename various freeze multixact variables. And I am

Re: [HACKERS] logical changeset generation v6

2013-09-25 Thread Steve Singer
On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit Rename various freeze multixact variables. And I am even partially guilty for that patch... Rebased patches attached. While testing the logical replication changes against my WIP logical

Re: [HACKERS] logical changeset generation v6

2013-09-25 Thread Andres Freund
On 2013-09-25 11:01:44 -0400, Steve Singer wrote: On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit Rename various freeze multixact variables. And I am even partially guilty for that patch... Rebased patches attached. While testing the

Re: [HACKERS] logical changeset generation v6

2013-09-25 Thread Steve Singer
On 09/25/2013 11:08 AM, Andres Freund wrote: On 2013-09-25 11:01:44 -0400, Steve Singer wrote: On 09/17/2013 10:31 AM, Andres Freund wrote: This patch set now fails to apply because of the commit Rename various freeze multixact variables. And I am even partially guilty for that patch...

Re: [HACKERS] logical changeset generation v6

2013-09-24 Thread Andres Freund
On 2013-09-23 23:12:53 -0400, Robert Haas wrote: What exactly is the purpose of this tool? My impression is that the output of logical replication is a series of function calls to a logical replication plugin, but does that plugin necessarily have to produce an output format that gets

Re: [HACKERS] logical changeset generation v6

2013-09-24 Thread Robert Haas
On Tue, Sep 24, 2013 at 4:15 AM, Andres Freund and...@2ndquadrant.com wrote: There needs to be a client acking the reception of the data in some form. There's currently two output methods, SQL and walstreamer, but there easily could be further, it's basically two functions you have write.

Re: [HACKERS] logical changeset generation v6

2013-09-24 Thread Andres Freund
On 2013-09-24 11:04:06 -0400, Robert Haas wrote: - Requiring a client is a short-sighted design. There's no reason we shouldn't *support* having a client, but IMHO it shouldn't be the only way to use the feature. There really aren't many limitations preventing you from doing anything else.

Re: [HACKERS] logical changeset generation v6

2013-09-24 Thread Steve Singer
On 09/24/2013 11:21 AM, Andres Freund wrote: Not having a consumer of the walsender interface included sounds like a bad idea to me, even if it were only useful for testing. Now, you could argue it should be in /contrib - and I wouldn't argue against that except it shares code with the rest of

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Andres Freund
On 2013-09-20 14:15:23 -0700, Peter Geoghegan wrote: I have a little bit of feedback that I forgot to mention in my earlier reviews, because I thought it was too trivial then: something about the name pg_receivellog annoys me in a way that the name pg_receivexlog does not. Specifically, it

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Peter Geoghegan
On Mon, Sep 23, 2013 at 1:46 AM, Andres Freund and...@2ndquadrant.com wrote: pg_receivelogical? Protest now or forever hold your peace. I was thinking pg_receiveloglog, but that works just as well. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Andres Freund
On 2013-09-23 13:47:05 -0300, Alvaro Herrera wrote: Andres Freund escribió: On 2013-09-20 14:15:23 -0700, Peter Geoghegan wrote: I have a little bit of feedback that I forgot to mention in my earlier reviews, because I thought it was too trivial then: something about the name

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Alvaro Herrera
Andres Freund escribió: On 2013-09-20 14:15:23 -0700, Peter Geoghegan wrote: I have a little bit of feedback that I forgot to mention in my earlier reviews, because I thought it was too trivial then: something about the name pg_receivellog annoys me in a way that the name pg_receivexlog

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Peter Geoghegan
On Mon, Sep 23, 2013 at 9:54 AM, Andres Freund and...@2ndquadrant.com wrote: I still find it wierd/inconsistent to have: * pg_receivexlog * pg_recvlogical binaries, even from the same source directory. Why once pg_recv and once pg_receive? +1 -- Peter Geoghegan -- Sent via pgsql-hackers

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Alvaro Herrera
Andres Freund escribió: On 2013-09-23 13:47:05 -0300, Alvaro Herrera wrote: I had proposed pg_recvlogical I still find it wierd/inconsistent to have: * pg_receivexlog * pg_recvlogical binaries, even from the same source directory. Why once pg_recv and once pg_receive? Well. What are

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Peter Eisentraut
On 9/23/13 12:54 PM, Andres Freund wrote: I still find it wierd/inconsistent to have: * pg_receivexlog * pg_recvlogical binaries, even from the same source directory. Why once pg_recv and once pg_receive? It's consistent because they are the same length! (Obviously, this would severely

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Robert Haas
On Mon, Sep 23, 2013 at 1:11 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Andres Freund escribió: On 2013-09-23 13:47:05 -0300, Alvaro Herrera wrote: I had proposed pg_recvlogical I still find it wierd/inconsistent to have: * pg_receivexlog * pg_recvlogical binaries, even from the

Re: [HACKERS] logical changeset generation v6

2013-09-23 Thread Peter Geoghegan
On Mon, Sep 23, 2013 at 8:12 PM, Robert Haas robertmh...@gmail.com wrote: The existence of a tool like pg_receivellog seems to presuppose that the goal is spit out logical change records as text, but I'm not sure that's actually going to be a very common thing to want to do... Sure, but I

Re: [HACKERS] logical changeset generation v6

2013-09-21 Thread Steve Singer
On 09/20/2013 06:33 AM, Andres Freund wrote: Hi, The points I find daunting are the semantics, like: * How do we control whether a standby is allowed prevent WAL file removal. What if archiving is configured? * How do we control whether a standby is allowed to peg xmin? * How long do we

Re: [HACKERS] logical changeset generation v6

2013-09-20 Thread Andres Freund
Hi, On 2013-09-19 12:05:35 -0400, Robert Haas wrote: No question. I'm not saying that that optimization shouldn't go in right after the main patch does, but IMHO right now there are too many things going in the 0004 patch to discuss them all simultaneously. I'd like to find a way of

Re: [HACKERS] logical changeset generation v6

2013-09-20 Thread Peter Geoghegan
On Thu, Sep 19, 2013 at 7:43 AM, Andres Freund and...@2ndquadrant.com wrote: More generally, the thing that bugs me about this approach is that logical replication is not really special, but what you've done here MAKES it special. There are plenty of other situations where we are too

Re: [HACKERS] logical changeset generation v6

2013-09-19 Thread Robert Haas
On Tue, Sep 17, 2013 at 10:31 AM, Andres Freund and...@2ndquadrant.com wrote: Rebased patches attached. I spent a bit of time looking at these patches yesterday and today. It seems to me that there's a fair amount of stylistic cleanup that is still needed, and some pretty bad naming choices, and

Re: [HACKERS] logical changeset generation v6

2013-09-19 Thread Andres Freund
Hi Robert, On 2013-09-19 10:02:31 -0400, Robert Haas wrote: On Tue, Sep 17, 2013 at 10:31 AM, Andres Freund and...@2ndquadrant.com wrote: Rebased patches attached. I spent a bit of time looking at these patches yesterday and today. It seems to me that there's a fair amount of stylistic

Re: [HACKERS] logical changeset generation v6

2013-09-19 Thread Robert Haas
On Thu, Sep 19, 2013 at 10:43 AM, Andres Freund and...@2ndquadrant.com wrote: - Looking specifically at the 0004 patch, I think that the RecentGlobalDataXmin changes are logically separate from the rest of the patch, and that if we're going to commit them at all, it should be separate from the

Re: [HACKERS] logical changeset generation v6

2013-09-18 Thread Fujii Masao
On Tue, Sep 17, 2013 at 11:31 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-09-17 09:45:28 -0400, Peter Eisentraut wrote: On 9/15/13 11:30 AM, Andres Freund wrote: On 2013-09-15 11:20:20 -0400, Peter Eisentraut wrote: On Sat, 2013-09-14 at 22:49 +0200, Andres Freund wrote:

Re: [HACKERS] logical changeset generation v6

2013-09-17 Thread Peter Eisentraut
On 9/15/13 11:30 AM, Andres Freund wrote: On 2013-09-15 11:20:20 -0400, Peter Eisentraut wrote: On Sat, 2013-09-14 at 22:49 +0200, Andres Freund wrote: Attached you can find the newest version of the logical changeset generation patchset. You probably have bigger things to worry about, but

Re: [HACKERS] logical changeset generation v6

2013-09-15 Thread Peter Eisentraut
What's with 0001-Improve-regression-test-for-8410.patch? Did you mean to include that? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] logical changeset generation v6

2013-09-15 Thread Andres Freund
On 2013-09-15 10:03:54 -0400, Peter Eisentraut wrote: What's with 0001-Improve-regression-test-for-8410.patch? Did you mean to include that? Gah, no. That's already committed and unrelated. Stupid wildcard. Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] logical changeset generation v6

2013-09-15 Thread Peter Eisentraut
On Sat, 2013-09-14 at 22:49 +0200, Andres Freund wrote: Attached you can find the newest version of the logical changeset generation patchset. You probably have bigger things to worry about, but please check the results of cpluspluscheck, because some of the header files don't include header

Re: [HACKERS] logical changeset generation v6

2013-09-15 Thread Andres Freund
On 2013-09-15 11:20:20 -0400, Peter Eisentraut wrote: On Sat, 2013-09-14 at 22:49 +0200, Andres Freund wrote: Attached you can find the newest version of the logical changeset generation patchset. You probably have bigger things to worry about, but please check the results of