Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-02-04 Thread Robert Haas
On Sat, Feb 2, 2013 at 4:38 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-28 16:55:52 -0500, Steve Singer wrote: If your using non-surragate /natural primary keys this tends to come up occasionally due to data-entry errors or renames. I'm looking at this from the point of view

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-02-02 Thread Andres Freund
On 2013-01-28 16:55:52 -0500, Steve Singer wrote: If your using non-surragate /natural primary keys this tends to come up occasionally due to data-entry errors or renames. I'm looking at this from the point of view of what do I need to use this as a source for a production replication system

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-28 Thread Andres Freund
On 2013-01-26 16:20:33 -0500, Steve Singer wrote: On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-28 Thread Andres Freund
Hi, On 2013-01-27 23:07:51 -0500, Steve Singer wrote: A few more comments; In decode.c DecodeDelete + if (r-xl_len = (SizeOfHeapDelete + SizeOfHeapHeader)) + { + elog(DEBUG2, huh, no primary key for a delete on wal_level = logical?); + return; + } + I think we

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-28 Thread Steve Singer
On 13-01-28 06:17 AM, Andres Freund wrote: Hi, 3. Pass the delete (with no key values) onto the replication client and let it deal with it (see 1 and 2) Hm. While I agree that nicer behaviour would be good I think the real enforcement should happen on a higher level, e.g. with event triggers

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-27 Thread Steve Singer
On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-26 Thread Steve Singer
On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Simon Riggs
On 24 January 2013 01:17, Robert Haas robertmh...@gmail.com wrote: I agree. The thing that scares me about the logical replication stuff is not that it might be slow (and if your numbers are to be believed, it isn't), but that I suspect it's riddled with bugs and possibly some questionable

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Heikki Linnakangas
On 24.01.2013 00:30, Andres Freund wrote: Hi, I decided to reply on the patches thread to be able to find this later. On 2013-01-23 22:48:50 +0200, Heikki Linnakangas wrote: logical changeset generation v4 This is a boatload of infrastructure for supporting logical replication, yet we have no

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Andres Freund
Hi Robert, Hi all, On 2013-01-23 20:17:04 -0500, Robert Haas wrote: On Wed, Jan 23, 2013 at 5:30 PM, Andres Freund and...@2ndquadrant.com wrote: The only reason the submitted version of logical decoding is comparatively slow is that its xmin update policy is braindamaged, working on that

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Heikki Linnakangas
One random thing that caught my eye in the patch, I though I'd mention it while I still remember: In heap_delete, you call heap_form_tuple() in a critical section. That's a bad idea, because if it runs out of memory - PANIC. - Heikki -- Sent via pgsql-hackers mailing list

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Andres Freund
On 2013-01-24 12:38:25 +0200, Heikki Linnakangas wrote: On 24.01.2013 00:30, Andres Freund wrote: Hi, I decided to reply on the patches thread to be able to find this later. On 2013-01-23 22:48:50 +0200, Heikki Linnakangas wrote: logical changeset generation v4 This is a boatload of

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Steve Singer
On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get there? I was able to get something that

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Robert Haas
On Thu, Jan 24, 2013 at 6:14 AM, Andres Freund and...@2ndquadrant.com wrote: Thats way much more along the lines of what I am afraid of than the performance stuff - but Heikki cited those, so I replied to that. Note that I didn't say this must, must go in - I just don't think Heikki's

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Now, the bad news is, I don't think it's very reasonable to try to commit this to 9.3. I think it is just too much stuff too late in the cycle. I've reviewed some of the patches from time to time but there is a lot more stuff and it's big and

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Heikki Linnakangas
On 24.01.2013 20:27, Robert Haas wrote: Before getting bogged down in technical commentary, let me say this very clearly: I am enormously grateful for your work on this project. Logical replication based on WAL decoding is a feature of enormous value that PostgreSQL has needed for a long time,

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Andres Freund
Hi! On 2013-01-24 13:27:00 -0500, Robert Haas wrote: On Thu, Jan 24, 2013 at 6:14 AM, Andres Freund and...@2ndquadrant.com wrote: Before getting bogged down in technical commentary, let me say this very clearly: I am enormously grateful for your work on this project. Logical replication

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Bruce Momjian
On Fri, Jan 25, 2013 at 02:16:09AM +0100, Andres Freund wrote: What I am afraid though is that it basically goes on like this in the next commitfests: * 9.4-CF1: no serious reviewer comments because they are busy doing release work * 9.4-CF2: all are relieved that the release is over and a

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-23 Thread Andres Freund
Hi, I decided to reply on the patches thread to be able to find this later. On 2013-01-23 22:48:50 +0200, Heikki Linnakangas wrote: logical changeset generation v4 This is a boatload of infrastructure for supporting logical replication, yet we have no code actually implementing logical

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-23 Thread Robert Haas
On Wed, Jan 23, 2013 at 5:30 PM, Andres Freund and...@2ndquadrant.com wrote: pgbench upstream: tps: 22275.941409 space overhead: 0% pgbench logical-submitted tps: 16274.603046 space overhead: 2.1% pgbench logical-HEAD (will submit updated version tomorrow or so): tps: 20853.341551 space

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-23 Thread Joshua D. Drake
On 01/23/2013 05:17 PM, Robert Haas wrote: Of course, I have no evidence that that will happen. But it is a really big piece of code, and therefore unless you are superman, it's probably got a really large number of bugs. The scary thing is that it is not as if we can say, well, this is a