Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-22 Thread Kevin Grittner
Simon Riggs wrote: Greg Stark st...@mit.edu wrote: Tom Lane t...@sss.pgh.pa.us wrote: Isn't there an even more serious problem, namely that this assumes *all* transactions are serializable? Do you mean in terms of the serializable transaction isolation level, or something else? I haven't

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-15 Thread Tatsuo Ishii
The design Andres and Simon have advanced already eliminates a lot of the common failure cases (now(), random(), nextval()) suffered by pgPool and similar tools. But remember, this feature doesn't have to be Well, pgpool-II already solved the now() case by using query rewriting technique. The

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Hannu Krosing
On 10/11/2012 04:31 AM, Tom Lane wrote: Greg Stark st...@mit.edu writes: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Isn't there an even more serious problem, namely that this assumes *all* transactions are serializable? What happens when they aren't? Or even just that

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Hannu Krosing
On 10/11/2012 03:10 AM, Robert Haas wrote: On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: The purpose of ApplyCache/transaction reassembly is to reassemble interlaced records, and organise them by XID, so that the consumer client code sees only streams (well,

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Josh Berkus
On 10/10/12 7:26 PM, Bruce Momjian wrote: How does Slony write its changes without causing serialization replay conflicts? Since nobody from the Slony team answered this: a) Slony replicates *rows*, not *statements* b) Slony uses serializable mode internally for row replication c) it's

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Christopher Browne
On Wed, Oct 10, 2012 at 10:26 PM, Bruce Momjian br...@momjian.us wrote: How does Slony write its changes without causing serialization replay conflicts? It uses a sequence to break any ordering conflicts at the time that data is inserted into its log tables. If there are two transactions, A

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Simon Riggs
On 11 October 2012 03:16, Greg Stark st...@mit.edu wrote: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think I've mentioned it before, but in the interest of not being seen to critique the bikeshed only after it's been painted: this design gives up something very

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-11 Thread Steve Singer
On 12-10-11 06:27 PM, Josh Berkus wrote: On 10/10/12 7:26 PM, Bruce Momjian wrote: How does Slony write its changes without causing serialization replay conflicts? Since nobody from the Slony team answered this: a) Slony replicates *rows*, not *statements* True, but the proposed logical

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Peter Geoghegan
On 15 September 2012 01:39, Andres Freund and...@2ndquadrant.com wrote: (0008-Introduce-wal-decoding-via-catalog-timetravel.patch) This patch is the 8th of 8 in a patch series that covers different aspects of the bi-directional replication feature planned for PostgreSQL 9.3. For those that are

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Thu, Oct 11, 2012 at 12:02:26AM +0100, Peter Geoghegan wrote: On 15 September 2012 01:39, Andres Freund and...@2ndquadrant.com wrote: (0008-Introduce-wal-decoding-via-catalog-timetravel.patch) This patch is the 8th of 8 in a patch series that covers different aspects of the

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Josh Berkus
Does this design allow replication/communcation between clusters running different major versions of Postgres? Just catching up on your email, hmmm? Yes, that's part of the design 2Q presented. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Thu, Oct 11, 2012 at 01:34:58AM +0200, anara...@anarazel.de wrote: Bruce Momjian br...@momjian.us schrieb: On Thu, Oct 11, 2012 at 12:02:26AM +0100, Peter Geoghegan wrote: On 15 September 2012 01:39, Andres Freund and...@2ndquadrant.com wrote:

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Robert Haas
On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: The purpose of ApplyCache/transaction reassembly is to reassemble interlaced records, and organise them by XID, so that the consumer client code sees only streams (well, lists) of records split by XID. I think I've

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Wed, Oct 10, 2012 at 09:10:48PM -0400, Robert Haas wrote: You consider this to be a throw-away function that won't ever be committed. However, I strongly feel that you should move it into /contrib, so that it can serve as a sort of reference implementation for authors of decoder client

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: The purpose of ApplyCache/transaction reassembly is to reassemble interlaced records, and organise them by XID, so that the consumer client code sees only streams (well,

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Greg Stark
On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think I've mentioned it before, but in the interest of not being seen to critique the bikeshed only after it's been painted: this design gives up something very important that exists in our current built-in replication

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
Hi, First of: Awesome review. On Thursday, October 11, 2012 01:02:26 AM Peter Geoghegan wrote: What follows is an initial overview of the patch (or at least my understanding of the patch, which you may need to correct), and some points of concern. * applycache module which reassembles

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Bruce Momjian
On Thu, Oct 11, 2012 at 03:16:39AM +0100, Greg Stark wrote: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think I've mentioned it before, but in the interest of not being seen to critique the bikeshed only after it's been painted: this design gives up something very

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Isn't there an even more serious problem, namely that this assumes *all* transactions are serializable? What happens when they aren't? Or even just that the effective commit order is not XID

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 03:10:48 AM Robert Haas wrote: On Wed, Oct 10, 2012 at 7:02 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: The purpose of ApplyCache/transaction reassembly is to reassemble interlaced records, and organise them by XID, so that the consumer client code sees

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 04:16:39 AM Greg Stark wrote: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: I think I've mentioned it before, but in the interest of not being seen to critique the bikeshed only after it's been painted: this design gives up something very

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 04:31:21 AM Tom Lane wrote: Greg Stark st...@mit.edu writes: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Isn't there an even more serious problem, namely that this assumes *all* transactions are serializable? What happens when they

Re: [HACKERS] [PATCH 8/8] Introduce wal decoding via catalog timetravel

2012-10-10 Thread Andres Freund
On Thursday, October 11, 2012 04:49:20 AM Andres Freund wrote: On Thursday, October 11, 2012 04:31:21 AM Tom Lane wrote: Greg Stark st...@mit.edu writes: On Thu, Oct 11, 2012 at 2:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Isn't there an even more serious problem, namely that this assumes