Re: [HACKERS] Issues with two-server Synch Rep

2010-10-19 Thread Simon Riggs
On Tue, 2010-10-19 at 09:36 -0700, Josh Berkus wrote: > >> Absolutely. For a synch standby, you can't tolerate any standby delay > >> at all. This means that anywhere from 1/4 to 3/4 of queries on the > >> standby would be cancelled on any high-traffic OLTP server. Hence, > >> "useless". > > > >

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-19 Thread Josh Berkus
Absolutely. For a synch standby, you can't tolerate any standby delay at all. This means that anywhere from 1/4 to 3/4 of queries on the standby would be cancelled on any high-traffic OLTP server. Hence, "useless". Don't agree with your numbers there and you seem to be assuming no workaroun

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-14 Thread Dimitri Fontaine
Peter Eisentraut writes: > Seems like a reasonable solution, although I've never heard > of .emacs.custom; my emacsen have always written their custom settings > somewhere in the middle of the .emacs proper. See (info "(emacs) Saving Customizations") and add the following to your setup: (set

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-13 Thread Simon Riggs
On Mon, 2010-10-11 at 11:07 -0700, Josh Berkus wrote: > Absolutely. For a synch standby, you can't tolerate any standby delay > at all. This means that anywhere from 1/4 to 3/4 of queries on the > standby would be cancelled on any high-traffic OLTP server. Hence, > "useless". Don't agree with

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-13 Thread Simon Riggs
On Thu, 2010-10-07 at 11:05 -0700, Josh Berkus wrote: > Simon, Fujii, > > What follows are what I see as the major issues with making two-server > synch replication work well. I would like to have you each answer them, > explaining how your patch and your design addresses each issue. I > believe

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-13 Thread Peter Eisentraut
On mån, 2010-10-11 at 18:44 -0700, Greg Stark wrote: > So we've been over this. All the pieces you need are already there: > you can handle this without any nasty comment grunging by just writing > the new setting to a postgresql.auto and including that from > postgresql.conf. Include a note in pos

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-13 Thread Fujii Masao
On Fri, Oct 8, 2010 at 3:05 AM, Josh Berkus wrote: > Adding a Synch Standby > --- > What is the procedure for adding a new synchronous standby in your > implementation?  That is, how do we go from having a standby server with > an empty PGDATA to having a working synchronous st

SQL command to edit postgresql.conf, with comments (was: [HACKERS] Issues with two-server Synch Rep)

2010-10-12 Thread Dimitri Fontaine
Robert Haas writes: > That's probably not going to happen until we have a way to update > postgresql.conf via SQL. Which, I maintain, as I have maintained > before, is not going to happen until we get rid of the comments, > because otherwise absolutely any implementation anyone proposes will > ge

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 10:17 PM, Gurjeet Singh wrote: >> > Yes.  I had the idea of trying to fix this by allowing the standby to >> > retain old versions of entire pages that got cleaned up on the master, >> > until the transactions that might want to read the old pages were >> > gone.  But that

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Gurjeet Singh
On Mon, Oct 11, 2010 at 7:16 PM, Jeff Davis wrote: > On Mon, 2010-10-11 at 16:07 -0400, Robert Haas wrote: > > >> It would be far better if we could decouple master cleanup from > > >> standby cleanup, so that only the machine that actually has the old > > >> query gets bloated. However, no one

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 9:44 PM, Greg Stark wrote: > On Mon, Oct 11, 2010 at 5:41 PM, Robert Haas wrote: >> That's probably not going to happen until we have a way to update >> postgresql.conf via SQL.  Which, I maintain, as I have maintained >> before, is not going to happen until we get rid of

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Greg Stark
On Mon, Oct 11, 2010 at 5:41 PM, Robert Haas wrote: > That's probably not going to happen until we have a way to update > postgresql.conf via SQL.  Which, I maintain, as I have maintained > before, is not going to happen until we get rid of the comments, > because otherwise absolutely any implemen

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 9:29 PM, Josh Berkus wrote: What is your source for those numbers?  They could be right, but I simply don't know. >>> pg_bench tests with asynch rep and standby_delay = 0.  Not rigorous, but >>> enough to show that there is a problem there.  Doing pg_bench with a

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Josh Berkus
> That's probably not going to happen until we have a way to update > postgresql.conf via SQL. Which, I maintain, as I have maintained > before, is not going to happen until we get rid of the comments, > because otherwise absolutely any implementation anyone proposes will > get criticized for fai

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 7:16 PM, Jeff Davis wrote: > On Mon, 2010-10-11 at 16:07 -0400, Robert Haas wrote: >> > I was initially taken aback by the word "useless" as well. However, I >> > had trouble thinking of a use case that isn't better solved by sync rep >> > without HS, or async rep. I don't

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 5:40 PM, Josh Berkus wrote: >> Obviously.  I presume it'll be something like "update postgresql.conf >> or recovery.conf and run pg_ctl reload", but I haven't (yet, anyway) >> verified the actual behavior of the patches, but if the above isn't >> feasible then we have a pro

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Jeff Davis
On Mon, 2010-10-11 at 16:07 -0400, Robert Haas wrote: > > I was initially taken aback by the word "useless" as well. However, I > > had trouble thinking of a use case that isn't better solved by sync rep > > without HS, or async rep. I don't have the numbers either though, so > > perhaps someone do

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Josh Berkus
> Obviously. I presume it'll be something like "update postgresql.conf > or recovery.conf and run pg_ctl reload", but I haven't (yet, anyway) > verified the actual behavior of the patches, but if the above isn't > feasible then we have a problem. Right. That's why I asked the question. Mind yo

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 4:01 PM, Jeff Davis wrote: > On Mon, 2010-10-11 at 15:22 -0400, Robert Haas wrote: >> >> This is a completely separate issue from making replication >> >> synchronous.  And, really?  Useless for running read queries? >> > >> > Absolutely.  For a synch standby, you can't tol

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Jeff Davis
On Mon, 2010-10-11 at 15:22 -0400, Robert Haas wrote: > >> This is a completely separate issue from making replication > >> synchronous. And, really? Useless for running read queries? > > > > Absolutely. For a synch standby, you can't tolerate any standby delay > > at all. This means that anywh

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Mon, Oct 11, 2010 at 2:07 PM, Josh Berkus wrote: >> I'll take a crack at answering these.  I don't think that the >> procedure for setting up a standby server is going to change much. >> The idea is presumably that you set up an async standby more or less >> as you do now and then make whatever

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Josh Berkus
Robert, > I'll take a crack at answering these. I don't think that the > procedure for setting up a standby server is going to change much. > The idea is presumably that you set up an async standby more or less > as you do now and then make whatever configuration changes are > necessary to flip i

Re: [HACKERS] Issues with two-server Synch Rep

2010-10-11 Thread Robert Haas
On Thu, Oct 7, 2010 at 2:05 PM, Josh Berkus wrote: > What is the procedure for adding a new synchronous standby in your > implementation?  That is, how do we go from having a standby server with > an empty PGDATA to having a working synchronous standby? I'll take a crack at answering these. I do

[HACKERS] Issues with two-server Synch Rep

2010-10-07 Thread Josh Berkus
Simon, Fujii, What follows are what I see as the major issues with making two-server synch replication work well. I would like to have you each answer them, explaining how your patch and your design addresses each issue. I believe this will go a long way towards helping the majority of the commu