Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Andres Freund
On 2014-08-19 21:47:26 +0800, Craig Ringer wrote: > On 08/19/2014 06:21 PM, Andres Freund wrote: > > What's the problem with the COMMIT WITH (report_lsn on) I've proposed? > > Reporting the LSN in the command tag? Anything doing transparent > > failover needs to be aware of transaction boundaries a

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Craig Ringer
On 08/19/2014 06:21 PM, Andres Freund wrote: > What's the problem with the COMMIT WITH (report_lsn on) I've proposed? > Reporting the LSN in the command tag? Anything doing transparent > failover needs to be aware of transaction boundaries anyway. Tom's objection to a GUC applies there too - a cl

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Fujii Masao
On Tue, Aug 19, 2014 at 8:22 PM, Andres Freund wrote: > On 2014-08-19 19:59:51 +0900, Fujii Masao wrote: >> On Tue, Aug 19, 2014 at 7:21 PM, Andres Freund >> wrote: >> > On 2014-08-19 08:21:10 +0800, Craig Ringer wrote: >> >> On 08/19/2014 01:03 AM, Robert Haas wrote: >> >> > 2. I agree that it'

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Andres Freund
On 2014-08-19 19:59:51 +0900, Fujii Masao wrote: > On Tue, Aug 19, 2014 at 7:21 PM, Andres Freund wrote: > > On 2014-08-19 08:21:10 +0800, Craig Ringer wrote: > >> On 08/19/2014 01:03 AM, Robert Haas wrote: > >> > 2. I agree that it's not good to have this get controlled by a GUC. > >> > If the be

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Fujii Masao
On Tue, Aug 19, 2014 at 7:21 PM, Andres Freund wrote: > On 2014-08-19 08:21:10 +0800, Craig Ringer wrote: >> On 08/19/2014 01:03 AM, Robert Haas wrote: >> > 2. I agree that it's not good to have this get controlled by a GUC. >> > If the behavior change is big enough that it's going to break client

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Andres Freund
On 2014-08-19 08:21:10 +0800, Craig Ringer wrote: > On 08/19/2014 01:03 AM, Robert Haas wrote: > > 2. I agree that it's not good to have this get controlled by a GUC. > > If the behavior change is big enough that it's going to break clients, > > adding a GUC isn't a sufficient remedy. If it's not,

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Greg Stark
On Tue, Aug 19, 2014 at 1:21 AM, Craig Ringer wrote: > There's plenty of agreement on "not a GUC" - but what about alternatives? It could be a new protocol message. Currently there are no transaction oriented protocol messages (other than the "transaction status" in ReadyForQuery). But would it n

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Craig Ringer
On 08/19/2014 01:03 AM, Robert Haas wrote: > 2. I agree that it's not good to have this get controlled by a GUC. > If the behavior change is big enough that it's going to break clients, > adding a GUC isn't a sufficient remedy. If it's not, adding a GUC is > unnecessary. There's plenty of agreeme

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Greg Stark
On 18 Aug 2014 20:05, "Greg Stark" wrote: >Having it in the commit message guarantees the client never has to deal with strange states like " I know this transaction committed but I know when" Sigh. Typing on the phone. "But I *don't* know when"

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Greg Stark
On 15 Aug 2014 14:54, "Tom Lane" wrote: > > Andres Freund writes: > > On 2014-08-14 12:21:38 -0400, Robert Haas wrote: > >> And what does that actually do? Send back a result-set, or a new > >> protocol message? > > > What I was thinking of was to return "COMMIT X/X" instead of > > "COMMIT". Sin

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Alvaro Herrera
Robert Haas wrote: > Moreover, even if somebody is (rather oddly, IMV) checking for > command_tag == "COMMIT", it's not going to be a difficult change to > check for command_tag == "COMMIT" or substr(command_tag,0,7) == > "COMMIT ". That's a level of incompatibility that would certainly > deserve

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Robert Haas
On Mon, Aug 18, 2014 at 12:49 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Aug 15, 2014 at 9:54 AM, Tom Lane wrote: >>> Au contraire: it will break any piece of code that is expecting a COMMIT >>> command tag to look like exactly "COMMIT" and not "COMMIT something". > >> Well, I remember

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 15, 2014 at 9:54 AM, Tom Lane wrote: >> Au contraire: it will break any piece of code that is expecting a COMMIT >> command tag to look like exactly "COMMIT" and not "COMMIT something". > Well, I remember debating this with you once before, when we were > decidi

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Robert Haas
On Fri, Aug 15, 2014 at 9:54 AM, Tom Lane wrote: > Andres Freund writes: >> On 2014-08-14 12:21:38 -0400, Robert Haas wrote: >>> And what does that actually do? Send back a result-set, or a new >>> protocol message? > >> What I was thinking of was to return "COMMIT X/X" instead of >> "COMMIT". S

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-15 Thread Andres Freund
On 2014-08-15 09:54:01 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-08-14 12:21:38 -0400, Robert Haas wrote: > >> And what does that actually do? Send back a result-set, or a new > >> protocol message? > > > What I was thinking of was to return "COMMIT X/X" instead of > > "COMMIT".

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-15 Thread Tom Lane
Andres Freund writes: > On 2014-08-14 12:21:38 -0400, Robert Haas wrote: >> And what does that actually do? Send back a result-set, or a new >> protocol message? > What I was thinking of was to return "COMMIT X/X" instead of > "COMMIT". Since that's only sent when COMMIT WITH (report_commit_lsn

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-15 Thread Andres Freund
On 2014-08-14 12:21:38 -0400, Robert Haas wrote: > On Sat, Aug 9, 2014 at 12:54 PM, Andres Freund wrote: > > I don't think we really need to embed it at that level. And it doesn't > > have to be always on - only clients that ask for it need to get the > > answer. Something like COMMIT WITH (report

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-14 Thread Josh Berkus
On 08/14/2014 05:45 PM, Craig Ringer wrote: > Wouldn't that force client drivers - libpq, psqlODBC, PgJDBC, etc - to > all watch for explicit "COMMIT"s sent by the application and rewrite them? Realistically, users are going to need new drivers to take advantage of any automated connection failov

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-14 Thread Craig Ringer
On 08/15/2014 12:21 AM, Robert Haas wrote: > On Sat, Aug 9, 2014 at 12:54 PM, Andres Freund wrote: > I don't have a very clear idea whether this is a good idea in any form > because I can't quite imagine how this is going to be used by the > client without adding an unwarranted amount of complexi

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-14 Thread Robert Haas
On Sat, Aug 9, 2014 at 12:54 PM, Andres Freund wrote: > On 2014-08-07 21:02:54 -0400, Tom Lane wrote: >> Craig Ringer writes: >> > On 08/08/2014 03:54 AM, Tom Lane wrote: >> >> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >> >> at all. What happens five years from now

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-10 Thread Andres Freund
On 2014-08-10 08:50:58 +0800, Craig Ringer wrote: > On 08/10/2014 12:54 AM, Andres Freund wrote: > > On 2014-08-07 21:02:54 -0400, Tom Lane wrote: > >> Craig Ringer writes: > >>> On 08/08/2014 03:54 AM, Tom Lane wrote: > FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-09 Thread Craig Ringer
On 08/10/2014 12:54 AM, Andres Freund wrote: > On 2014-08-07 21:02:54 -0400, Tom Lane wrote: >> Craig Ringer writes: >>> On 08/08/2014 03:54 AM, Tom Lane wrote: FWIW, I think it's a seriously bad idea to expose LSNs in the protocol at all. What happens five years from now when we switc

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-09 Thread Andres Freund
On 2014-08-07 21:02:54 -0400, Tom Lane wrote: > Craig Ringer writes: > > On 08/08/2014 03:54 AM, Tom Lane wrote: > >> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol > >> at all. What happens five years from now when we switch to some other > >> implementation that doesn't

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 09:51 AM, Tom Lane wrote: >> AFAIK we don't _have_ a fancy negotiation system in the protocol, with >> back-and-forth exchanges of capabilities information. > > Maybe it's time to invent that. It would be positively foolish to > create any such behavior without a protocol version b

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Michael Paquier
On Fri, Aug 8, 2014 at 11:58 AM, Fujii Masao wrote: > On Fri, Aug 8, 2014 at 9:50 AM, Craig Ringer wrote: > ISTM that the proper solution to that problem is the introduction of > new synchronous replication mode which makes the transaction wait for > its WAL to be replayed by the standby. If this

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 10:58 AM, Fujii Masao wrote: > ISTM that the proper solution to that problem is the introduction of > new synchronous replication mode which makes the transaction wait for > its WAL to be replayed by the standby. If this mode is used, a client > doesn't need to track the LSN of each t

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Fujii Masao
On Fri, Aug 8, 2014 at 9:50 AM, Craig Ringer wrote: > On 08/08/2014 03:54 AM, Tom Lane wrote: >> Craig Ringer writes: >>> Hi all >>> To support transparent client-side failover in BDR, it's necessary to >>> know what the LSN of a node was at the time a transaction committed and >>> keep track of

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 09:02 AM, Tom Lane wrote: > Craig Ringer writes: >> On 08/08/2014 03:54 AM, Tom Lane wrote: >>> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >>> at all. What happens five years from now when we switch to some other >>> implementation that doesn't have LSNs

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/07/2014 11:42 PM, Robert Haas wrote: > I doubt whether it makes sense to do this without a broader > understanding of how the client-side failover mechanism would work. > If we're going to add something like this, it should include libpq > support for actually doing something useful with it.

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Tom Lane
Craig Ringer writes: > On 08/08/2014 03:54 AM, Tom Lane wrote: >> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >> at all. What happens five years from now when we switch to some other >> implementation that doesn't have LSNs? > Everyone who's relying on them already vi

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 03:54 AM, Tom Lane wrote: > Craig Ringer writes: >> Hi all >> To support transparent client-side failover in BDR, it's necessary to >> know what the LSN of a node was at the time a transaction committed and >> keep track of that in the client/proxy. > >> I'm thinking about adding a

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Tom Lane
Craig Ringer writes: > Hi all > To support transparent client-side failover in BDR, it's necessary to > know what the LSN of a node was at the time a transaction committed and > keep track of that in the client/proxy. > I'm thinking about adding a new message type in the protocol that gets > sent

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Robert Haas
On Wed, Aug 6, 2014 at 9:15 PM, Craig Ringer wrote: > To support transparent client-side failover in BDR, it's necessary to > know what the LSN of a node was at the time a transaction committed and > keep track of that in the client/proxy. I doubt whether it makes sense to do this without a broad

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Ants Aasma
On Thu, Aug 7, 2014 at 4:15 AM, Craig Ringer wrote: > I'm thinking about adding a new message type in the protocol that gets > sent immediately before CommandComplete, containing the LSN of the > commit. Clients would need to enable the sending of this message with a > GUC that they set when they

[HACKERS] Reporting the commit LSN at commit time

2014-08-06 Thread Craig Ringer
Hi all To support transparent client-side failover in BDR, it's necessary to know what the LSN of a node was at the time a transaction committed and keep track of that in the client/proxy. I'm thinking about adding a new message type in the protocol that gets sent immediately before CommandComple