Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Fujii Masao
On Fri, Oct 8, 2010 at 10:38 AM, Robert Haas robertmh...@gmail.com wrote: Yes, let's please just implement something simple and get it committed.  k = 1.  Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply) For my cases, I'm OK with this as the

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Yeb Havinga
Robert Haas wrote: Yes, let's please just implement something simple and get it committed. k = 1. Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply), SUSET so you can change it per txn. Done. We can revise it *the day after it's committed* if

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 4:29 AM, Yeb Havinga yebhavi...@gmail.com wrote: Robert Haas wrote: Yes, let's please just implement something simple and get it committed.  k = 1.  Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply), SUSET so you can

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Josh Berkus
On 10/07/2010 06:38 PM, Robert Haas wrote: Yes, let's please just implement something simple and get it committed. k = 1. Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply), SUSET so you can change it per txn. Done. We can revise it *the day

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-08 Thread Robert Haas
On Fri, Oct 8, 2010 at 12:29 PM, Josh Berkus j...@agliodbs.com wrote: On 10/07/2010 06:38 PM, Robert Haas wrote: Yes, let's please just implement something simple and get it committed.  k = 1.  Two GUCs (synchronous_standbys = name, name, name and synchronous_waitfor = none|recv|fsync|apply),

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Heikki Linnakangas
On 06.10.2010 19:26, Greg Smith wrote: Now, the more relevant question, what I actually need in order for a Sync Rep feature in 9.1 to be useful to the people who want it most I talk to. That would be a simple to configure setup where I list a subset of important nodes, and the appropriate

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Dave Page
On 10/7/10, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.10.2010 19:26, Greg Smith wrote: Now, the more relevant question, what I actually need in order for a Sync Rep feature in 9.1 to be useful to the people who want it most I talk to. That would be a simple to

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Josh Berkus
On 10/7/10 10:27 AM, Heikki Linnakangas wrote: The standby name is a GUC in the standby's configuration file: standby_name='bostonserver' The list of important nodes is also a GUC, in the master's configuration file: synchronous_standbys='bostonserver, oxfordserver' This seems to

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Robert Haas
On Thu, Oct 7, 2010 at 1:39 PM, Dave Page dp...@pgadmin.org wrote: On 10/7/10, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.10.2010 19:26, Greg Smith wrote: Now, the more relevant question, what I actually need in order for a Sync Rep feature in 9.1 to be useful to the

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Robert Haas
On Thu, Oct 7, 2010 at 1:45 PM, Josh Berkus j...@agliodbs.com wrote: On 10/7/10 10:27 AM, Heikki Linnakangas wrote: The standby name is a GUC in the standby's configuration file: standby_name='bostonserver' The list of important nodes is also a GUC, in the master's configuration file:

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Aidan Van Dyk
On Thu, Oct 7, 2010 at 1:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Let me check that I got this right, and add some details to make it more concrete: Each standby is given a name. It can be something like boston1 or testserver. It does *not* have to be unique across

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Josh Berkus
I think they work together fine. Greg's idea is that you list the important standbys, and a synchronization guarantee that you'd like to have for at least one of them. Simon's idea - at least at 10,000 feet - is that you can take a pass on that guarantee for transactions that don't need

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Robert Haas
On Thu, Oct 7, 2010 at 2:33 PM, Josh Berkus j...@agliodbs.com wrote: I think they work together fine.  Greg's idea is that you list the important standbys, and a synchronization guarantee that you'd like to have for at least one of them.  Simon's idea - at least at 10,000 feet - is that you

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Greg Stark
On Thu, Oct 7, 2010 at 10:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The standby name is a GUC in the standby's configuration file: standby_name='bostonserver' Fwiw I was hoping it would be possible to set every machine up with an identical postgresql.conf file. That

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Greg Smith
Josh Berkus wrote: This version of Standby Registration seems to add One More Damn Place You Need To Configure Standby (OMDPYNTCS) without adding any functionality you couldn't get *without* having a list on the master. Can someone explain to me what functionality is added by this approach vs.

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Robert Haas
On Thu, Oct 7, 2010 at 7:15 PM, Greg Smith g...@2ndquadrant.com wrote: Josh Berkus wrote: This version of Standby Registration seems to add One More Damn Place You Need To Configure Standby (OMDPYNTCS) without adding any functionality you couldn't get *without* having a list on the master.

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Heikki Linnakangas
On 07.10.2010 23:56, Greg Stark wrote: On Thu, Oct 7, 2010 at 10:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: The standby name is a GUC in the standby's configuration file: standby_name='bostonserver' Fwiw I was hoping it would be possible to set every machine up

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-07 Thread Heikki Linnakangas
On 07.10.2010 21:33, Josh Berkus wrote: 1) This version of Standby Registration seems to add One More Damn Place You Need To Configure Standby (OMDPYNTCS) without adding any functionality you couldn't get *without* having a list on the master. Can someone explain to me what functionality is

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-06 Thread Greg Smith
Josh Berkus wrote: However, I think we're getting way the heck away from how far we really want to go for 9.1. Can I point out to people that synch rep is going to involve a fair bit of testing and debugging, and that maybe we don't want to try to implement The World's Most Configurable

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-06 Thread Robert Haas
On Wed, Oct 6, 2010 at 12:26 PM, Greg Smith g...@2ndquadrant.com wrote: Now, the more relevant question, what I actually need in order for a Sync Rep feature in 9.1 to be useful to the people who want it most I talk to.  That would be a simple to configure setup where I list a subset of

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Markus Wanner
On 10/04/2010 11:32 PM, Robert Haas wrote: I think in the end this is not much different from standby registration; you still have registrations, they just represent groups of machines instead of single machines. Such groups are often easy to represent in CIDR notation, which would reduce the

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Mon, 2010-10-04 at 14:25 -0500, David Christensen wrote: Is there any benefit to be had from having standby roles instead of individual names? For instance, you could integrate this into quorum commit to express 3 of 5 reporting standbys, 1 berlin standby and 1 tokyo standby from a group

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Mon, 2010-10-04 at 12:45 -0700, Josh Berkus wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B) (C || D). Good point. Asking for quorum_commit = 3 would cover that requirement. Not exactly as

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Robert Haas
On Tue, Oct 5, 2010 at 8:34 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2010-10-04 at 12:45 -0700, Josh Berkus wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B) (C || D). Good point. Asking

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Tue, 2010-10-05 at 08:57 -0400, Robert Haas wrote: On Tue, Oct 5, 2010 at 8:34 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2010-10-04 at 12:45 -0700, Josh Berkus wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: Robert Haas wrote: Simon Riggs si...@2ndquadrant.com wrote: Josh Berkus wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B) (C || D). Good point. Asking for

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Markus Wanner
On 10/05/2010 04:07 PM, Kevin Grittner wrote: When you have one server functioning at each site you'll block until you get a third machine back, rather than replicating to both sites and remaining functional. That's not a very likely failure scenario, but yes. What if the admin wants to add a

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Tue, 2010-10-05 at 09:07 -0500, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: Robert Haas wrote: Simon Riggs si...@2ndquadrant.com wrote: Josh Berkus wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Robert Haas
On Tue, Oct 5, 2010 at 10:33 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-10-05 at 09:07 -0500, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: Robert Haas wrote: Simon Riggs si...@2ndquadrant.com wrote: Josh Berkus wrote: Quorum commit, even with configurable

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Tue, 2010-10-05 at 10:41 -0400, Robert Haas wrote: When you have one server functioning at each site you'll block until you get a third machine back, rather than replicating to both sites and remaining functional. And that is so important a consideration that you would like to move

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: Is it a common use case that people have more than 3 separate servers for one application, which is where the difference shows itself. I don't know how common it is, but we replicate circuit court data to two machines each at two sites. That way a

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Josh Berkus
Another check: does specifying replication by server in such detail mean we can't specify robustness at the transaction level? If we gave up that feature, it would be a greatloss for performance tuning. It's orthagonal. The kinds of configurations we're talking about simply define what it

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Tue, 2010-10-05 at 09:56 -0500, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: Is it a common use case that people have more than 3 separate servers for one application, which is where the difference shows itself. I don't know how common it is, but we replicate

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Robert Haas
On Tue, Oct 5, 2010 at 10:46 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-10-05 at 10:41 -0400, Robert Haas wrote: When you have one server functioning at each site you'll block until you get a third machine back, rather than replicating to both sites and remaining

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Tue, 2010-10-05 at 11:46 -0400, Robert Haas wrote: On Tue, Oct 5, 2010 at 10:46 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-10-05 at 10:41 -0400, Robert Haas wrote: When you have one server functioning at each site you'll block until you get a third machine back,

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Robert Haas
On Tue, Oct 5, 2010 at 12:40 PM, Simon Riggs si...@2ndquadrant.com wrote: Well, you only need to have the file at all on nodes you want to fail over to.  And aren't you going to end up rejiggering the config when you fail over anyway, based on what happened?  I mean, suppose you have three

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Simon Riggs
On Tue, 2010-10-05 at 10:41 -0400, Robert Haas wrote: Much of the engineering we are doing centers around use cases that are considerably more complex than what most people will do in real life. Why are we doing it then? What I have proposed behaves identically to Oracle Maximum Availability

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-05 Thread Robert Haas
On Tue, Oct 5, 2010 at 2:30 PM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-10-05 at 10:41 -0400, Robert Haas wrote: Much of the engineering we are doing centers around use cases that are considerably more complex than what most people will do in real life. Why are we doing it then?

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread Robert Haas
On Mon, Oct 4, 2010 at 3:08 AM, Markus Wanner mar...@bluegap.ch wrote: On 10/01/2010 05:06 PM, Dimitri Fontaine wrote: Wait forever can be done without standby registration, with quorum commit. Yeah, I also think the only reason for standby registration is ease of configuration (if at all).

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread Markus Wanner
On 10/04/2010 05:20 PM, Robert Haas wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B) (C || D). Good point. Can the proposed standby registration configuration format cover such a requirement? Regards

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread Robert Haas
On Mon, Oct 4, 2010 at 1:57 PM, Markus Wanner mar...@bluegap.ch wrote: On 10/04/2010 05:20 PM, Robert Haas wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B) (C || D). Good point. Can the proposed

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread David Christensen
On Oct 4, 2010, at 2:02 PM, Robert Haas wrote: On Mon, Oct 4, 2010 at 1:57 PM, Markus Wanner mar...@bluegap.ch wrote: On 10/04/2010 05:20 PM, Robert Haas wrote: Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B)

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread Mike Rylander
On Mon, Oct 4, 2010 at 3:25 PM, David Christensen da...@endpoint.com wrote: On Oct 4, 2010, at 2:02 PM, Robert Haas wrote: On Mon, Oct 4, 2010 at 1:57 PM, Markus Wanner mar...@bluegap.ch wrote: On 10/04/2010 05:20 PM, Robert Haas wrote: Quorum commit, even with configurable vote weights,

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread Josh Berkus
Quorum commit, even with configurable vote weights, can't handle a requirement that a particular commit be replicated to (A || B) (C || D). Good point. If this is the only feature which standby registration is needed for, has anyone written the code for it yet? Is anyone planning to? If

Re: [HACKERS] standby registration (was: is sync rep stalled?)

2010-10-04 Thread Robert Haas
On Mon, Oct 4, 2010 at 3:25 PM, David Christensen da...@endpoint.com wrote: On Oct 4, 2010, at 2:02 PM, Robert Haas wrote: On Mon, Oct 4, 2010 at 1:57 PM, Markus Wanner mar...@bluegap.ch wrote: On 10/04/2010 05:20 PM, Robert Haas wrote: Quorum commit, even with configurable vote weights,