Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-31 Thread Bruce Momjian
Robert Haas wrote: > On Sat, May 8, 2010 at 10:40 PM, Tom Lane wrote: > > Bruce Momjian writes: > >> Uh, did we decide that 'wal_keep_segments' was the best name for this > >> GUC setting? ?I know we shipped beta1 using that name. > > > > I thought min_wal_segments was a reasonable proposal, but

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-09 Thread Simon Riggs
On Sat, 2010-05-08 at 23:55 -0400, Robert Haas wrote: > On Sat, May 8, 2010 at 10:40 PM, Tom Lane wrote: > > Bruce Momjian writes: > >> Uh, did we decide that 'wal_keep_segments' was the best name for this > >> GUC setting? I know we shipped beta1 using that name. > > > > I thought min_wal_segme

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 10:40 PM, Tom Lane wrote: > Bruce Momjian writes: >> Uh, did we decide that 'wal_keep_segments' was the best name for this >> GUC setting?  I know we shipped beta1 using that name. > > I thought min_wal_segments was a reasonable proposal, but it wasn't > clear if there was

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-08 Thread Tom Lane
Bruce Momjian writes: > Uh, did we decide that 'wal_keep_segments' was the best name for this > GUC setting? I know we shipped beta1 using that name. I thought min_wal_segments was a reasonable proposal, but it wasn't clear if there was consensus or not. regards, tom lan

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-08 Thread Bruce Momjian
Bruce Momjian wrote: > Simon Riggs wrote: > > On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: > > > > > > > > (wal_keep_segments can be changed without restarting, right?) > > > > > > Should we allow -1 to mean "keep all segments"? > > > > Why is that not called "max_wal_segments"? wal_k

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-03 Thread Josh Berkus
> It's really both of those things, so we could call it > wal_min_keep_segments, but I think an even better name would be > bikeshed_segments. Speaking from my UI perspective, I don't think users will care what we call it. -- -- Josh Berkus

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-03 Thread Robert Haas
On Mon, May 3, 2010 at 2:54 PM, Kevin Grittner wrote: > Simon Riggs wrote: >> On Fri, 2010-04-30 at 13:41 -0500, Kevin Grittner wrote: > >>> Surely it would confuse people to see they have fewer than >>> min_wal_segments WAL segments. >> >> That does sound like a reasonable argument, though it al

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-03 Thread Kevin Grittner
Simon Riggs wrote: > On Fri, 2010-04-30 at 13:41 -0500, Kevin Grittner wrote: >> Surely it would confuse people to see they have fewer than >> min_wal_segments WAL segments. > > That does sound like a reasonable argument, though it also applies > to wal_keep_segments, so isn't an argument eithe

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-05-02 Thread Simon Riggs
On Fri, 2010-04-30 at 13:41 -0500, Kevin Grittner wrote: > Heikki Linnakangas wrote: > > > Yeah, min_wal_segments or something would make sense. > > Surely it would confuse people to see they have fewer than > min_wal_segments WAL segments. That does sound like a reasonable argument, though i

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Kevin Grittner
Heikki Linnakangas wrote: > Kevin Grittner wrote: >> Surely it would confuse people to see they have fewer than >> min_wal_segments WAL segments. > > they wouldn't see that, that's the point of the setting. I was thinking, in particular, about beginners poking around to see how things look afte

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Heikki Linnakangas
Kevin Grittner wrote: > Heikki Linnakangas wrote: > >> Yeah, min_wal_segments or something would make sense. > > Surely it would confuse people to see they have fewer than > min_wal_segments WAL segments. Umm, they wouldn't see that, that's the point of the setting. The segments are not remov

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Simon Riggs
On Fri, 2010-04-30 at 14:42 -0400, Tom Lane wrote: > Robert Haas writes: > > On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: > >> Why is that not called "max_wal_segments"? wal_keep_segments sounds like > >> its been through Google translate. > > > Because it's not a maximum? > > Indeed. I

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Tom Lane
Alvaro Herrera writes: > Bruce Momjian escribió: >> Which is where my 'wal_keep_segments = -1' idea came from. > Are you suggesting that -1 should mean "keep all segments that fit on > disk, but if creating a new segment fails with ENOSPC, recycle the > oldest one"? No, keep means keep. Even if

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Heikki Linnakangas
Michael Tharp wrote: > On 04/30/2010 01:53 PM, Robert Haas wrote: >> >> Well, one of us is. Why would you want to retain all of your WAL logs >> in pg_xlog forever? > > To create or re-synchronize SR slaves, one could change > wal_keep_segments to -1, run a backup, wait for the slaves to catch up

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Alvaro Herrera
Bruce Momjian escribió: > Which is where my 'wal_keep_segments = -1' idea came from. Are you suggesting that -1 should mean "keep all segments that fit on disk, but if creating a new segment fails with ENOSPC, recycle the oldest one"? -- Alvaro Herrerahttp://www.

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Tom Lane
Heikki Linnakangas writes: > Bruce Momjian wrote: >> Should we allow -1 to mean "keep all segments"? > Umm, you can't keep all segments around forever, can you? Surely you > have to recycle them sooner or later or you will run out of disk space. You couldn't use that as a permanent setting, but

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Robert Haas
On Fri, Apr 30, 2010 at 2:08 PM, Simon Riggs wrote: > On Fri, 2010-04-30 at 13:52 -0400, Robert Haas wrote: >> On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: >> > On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: >> >> > >> >> > (wal_keep_segments can be changed without restarting, rig

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Bruce Momjian
Heikki Linnakangas wrote: > Bruce Momjian wrote: > > Tom Lane wrote: > >> Heikki Linnakangas writes: > >>> Tom Lane wrote: > If you aren't archiving then there's no guarantee that you'll still have > a continuous WAL series starting from the start of the backup. > >>> I wasn't really thi

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Tom Lane
Robert Haas writes: > On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: >> Why is that not called "max_wal_segments"? wal_keep_segments sounds like >> its been through Google translate. > Because it's not a maximum? Indeed. It would really be more like min_wal_segments, if we wanted to name

Re: WAL page magic number (was Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct)

2010-04-30 Thread Heikki Linnakangas
Bruce Momjian wrote: > Heikki Linnakangas wrote: >> Tom Lane wrote: >>> Heikki Linnakangas writes: Browsing the history of XLOG_PAGE_MAGIC, it used to be incremented by one whenever the format changes, at least as long as it has been been in xlog_internal.h. It started at 0xD05B in

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Bruce Momjian
Heikki Linnakangas wrote: > Robert Haas wrote: > > On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: > >> On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: > (wal_keep_segments can be changed without restarting, right?) > >>> Should we allow -1 to mean "keep all segments"? > >> Why is

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Kevin Grittner
Heikki Linnakangas wrote: > Yeah, min_wal_segments or something would make sense. Surely it would confuse people to see they have fewer than min_wal_segments WAL segments. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: htt

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Heikki Linnakangas
Bruce Momjian wrote: > Tom Lane wrote: >> Heikki Linnakangas writes: >>> Tom Lane wrote: If you aren't archiving then there's no guarantee that you'll still have a continuous WAL series starting from the start of the backup. >>> I wasn't really thinking of this use case, but you could se

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Heikki Linnakangas
Robert Haas wrote: > On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: >> On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: (wal_keep_segments can be changed without restarting, right?) >>> Should we allow -1 to mean "keep all segments"? >> Why is that not called "max_wal_segments"? w

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Simon Riggs
On Fri, 2010-04-30 at 13:58 -0400, Bruce Momjian wrote: > Robert Haas wrote: > > On Fri, Apr 30, 2010 at 1:39 PM, Bruce Momjian wrote: > > > Robert Haas wrote: > > >> On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian wrote: > > >> > Tom Lane wrote: > > >> >> Heikki Linnakangas writes: > > >> >> >

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Simon Riggs
On Fri, 2010-04-30 at 13:52 -0400, Robert Haas wrote: > On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: > > On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: > >> > > >> > (wal_keep_segments can be changed without restarting, right?) > >> > >> Should we allow -1 to mean "keep all segment

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Apr 30, 2010 at 1:39 PM, Bruce Momjian wrote: > > Robert Haas wrote: > >> On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian wrote: > >> > Tom Lane wrote: > >> >> Heikki Linnakangas writes: > >> >> > Tom Lane wrote: > >> >> >> If you aren't archiving then there's no gua

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Michael Tharp
On 04/30/2010 01:53 PM, Robert Haas wrote: Well, one of us is. Why would you want to retain all of your WAL logs in pg_xlog forever? ...Robert To create or re-synchronize SR slaves, one could change wal_keep_segments to -1, run a backup, wait for the slaves to catch up, and change it back

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Robert Haas
On Fri, Apr 30, 2010 at 1:39 PM, Bruce Momjian wrote: > Robert Haas wrote: >> On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian wrote: >> > Tom Lane wrote: >> >> Heikki Linnakangas writes: >> >> > Tom Lane wrote: >> >> >> If you aren't archiving then there's no guarantee that you'll still >> >> >

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Robert Haas
On Fri, Apr 30, 2010 at 1:44 PM, Simon Riggs wrote: > On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: >> > >> > (wal_keep_segments can be changed without restarting, right?) >> >> Should we allow -1 to mean "keep all segments"? > > Why is that not called "max_wal_segments"? wal_keep_segmen

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Bruce Momjian
Simon Riggs wrote: > On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: > > > > > > (wal_keep_segments can be changed without restarting, right?) > > > > Should we allow -1 to mean "keep all segments"? > > Why is that not called "max_wal_segments"? wal_keep_segments sounds like > its been t

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Simon Riggs
On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: > > > > (wal_keep_segments can be changed without restarting, right?) > > Should we allow -1 to mean "keep all segments"? Why is that not called "max_wal_segments"? wal_keep_segments sounds like its been through Google translate. -- Simo

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian wrote: > > Tom Lane wrote: > >> Heikki Linnakangas writes: > >> > Tom Lane wrote: > >> >> If you aren't archiving then there's no guarantee that you'll still have > >> >> a continuous WAL series starting from the start of the ba

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Robert Haas
On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian wrote: > Tom Lane wrote: >> Heikki Linnakangas writes: >> > Tom Lane wrote: >> >> If you aren't archiving then there's no guarantee that you'll still have >> >> a continuous WAL series starting from the start of the backup. >> >> > I wasn't really t

Re: WAL page magic number (was Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct)

2010-04-30 Thread Bruce Momjian
Heikki Linnakangas wrote: > Tom Lane wrote: > > Heikki Linnakangas writes: > >> Browsing the history of XLOG_PAGE_MAGIC, it used to be incremented by > >> one whenever the format changes, at least as long as it has been been in > >> xlog_internal.h. It started at 0xD05B in 2005, and was incremente

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-30 Thread Bruce Momjian
Tom Lane wrote: > Heikki Linnakangas writes: > > Tom Lane wrote: > >> If you aren't archiving then there's no guarantee that you'll still have > >> a continuous WAL series starting from the start of the backup. > > > I wasn't really thinking of this use case, but you could set > > wal_keep_segmen

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 16:18 -0400, Tom Lane wrote: > * rename recovery_connections to hot_standby (and the underlying > variable from XLogRequestRecoveryConnections to something like > EnableHotStandby); OK > * change its default value to 'off'. OK -- Simon Riggs www.2ndQuadrant.co

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 4:18 PM, Tom Lane wrote: > This train is leaving the station. An enthusiastic +1 for train departure. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Tom Lane
Okay, this thread seems to be running out of steam, and we are running out of hours before the beta1 wrap. I am going to take it on my own authority to do the following: * rename recovery_connections to hot_standby (and the underlying variable from XLogRequestRecoveryConnections to something like

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> Yeah. ISTM the real bottom line here is that we have only a weak grasp >> on how these features will end up being used; or for that matter what >> the common error scenarios will be. I think that for the time being >> we should err on the side of b

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Kevin Grittner
Simon Riggs wrote: > [most significant extra WAL logging for hot standby is:] > * one extra WAL record every checkpoint, containing a full current > snapshot's worth of running xids 100-400 bytes typically, could go > up from there to 4000 bytes in very extreme write workloads that > also have m

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Tom Lane
"Kevin Grittner" writes: > Simon Riggs wrote: >> 9.0 without HS enabled is fairly untested on the standby, so it's >> not clear to me turning it off makes you any safer. > I think you've just made the strongest possible case for not > defaulting it on during beta testing. Yes. If we've had it

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Joshua D. Drake
On Thu, 2010-04-29 at 21:27 +0300, Heikki Linnakangas wrote: > I wasn't really following the join removal discussions, but it seems > much safer and less complex. And it would not have been feasible to have > both implementations of FSM around and have the user to choose. I have > been relieved b

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > What Heikki hasn't said is that setting max_prepared_transactions > incorrectly has always caused a fatal error at failover, and still does > so now when HS is disabled. Hmm, good point. That should probably be documented somewhere. Now that we WAL-log the max_prepared_xacts v

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 13:01 -0500, Kevin Grittner wrote: > Simon Riggs wrote: > > > I've asked for evidence that recovery is any slower as a result of > > HS. > > Can you quantify the impact on the number of bytes written to WAL > when switching from the archiving level to the hot standby leve

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-04-29 at 19:55 +0300, Heikki Linnakangas wrote: >> It's safer to run >> with recovery_connections off if you don't need the feature. > > Presumably your advice is also that people should not run with Streaming > Replication if they don't need that feature? Umm,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Kevin Grittner
Simon Riggs wrote: > 9.0 without HS enabled is fairly untested on the standby, so it's > not clear to me turning it off makes you any safer. I think you've just made the strongest possible case for not defaulting it on during beta testing. -Kevin -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 13:20 -0400, Robert Haas wrote: > This is a pretty good argument, and Heikki's argument just upthread > that a mismatched max_connections setting could bollix things is an > even better one. So I'm now changing my mind and thinking this should > be off by default, also. Wha

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Tom Lane
Simon Riggs writes: > Can we keep the default=on during beta and collect evidence before > making a final decision at release? I'd be for doing that if it didn't have adverse effects on the use of pg_start_backup (which is where we started this thread). But it would be to get more testing, not b

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 19:55 +0300, Heikki Linnakangas wrote: > And Aidan and me are human > beings, included in "everybody". Yes, that was too loose. I dislike that kind of argument and I'm sorry that slipped in here. > You know that there is cases where it causes problems in the standby, > even

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Kevin Grittner
Simon Riggs wrote: > I've asked for evidence that recovery is any slower as a result of > HS. Can you quantify the impact on the number of bytes written to WAL when switching from the archiving level to the hot standby level? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > Can we keep the default=on during beta and collect evidence before > making a final decision at release? That's tempting to get more (inadvertent) testing of HS, but I don't think it would be fair to the beta testers. The expectation is that what's in beta is what's in the rel

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 12:16 -0500, Kevin Grittner wrote: > Simon Riggs wrote: > > > Which is what everybody has requested. > > You continue to use the term "everybody" rather loosely. Sorry, that was imprecise, I didn't mean to exaggerate. -- Simon Riggs www.2ndQuadrant.com --

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Joshua D. Drake
On Thu, 2010-04-29 at 13:03 -0400, Tom Lane wrote: > There is not only a performance penalty, but a reliability penalty. > Enabling these switches turns on a whole lot of code that, with all > due respect to those who have worked on it, is absolutely positively > guaranteed to be full of bugs. No

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Greg Smith
Aidan Van Dyk wrote: It's all about the path of least *suprise*. My "least suprise" would have been that a similar config I have now with my PITR slaves would pretty much still work, and not suddenly start accepting connections, and even worse, at some later date when I've already verified that

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 13:03 -0400, Tom Lane wrote: > There is not only a performance penalty I've asked for evidence that recovery is any slower as a result of HS. If we had some, I'm guessing we'd be tuning rather than discussing this. > Enabling these switches turns on a whole lot of code that

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 1:03 PM, Tom Lane wrote: >>> That said, I'ld probalby be happy with PG 9 having a "default" config >>> of: >>>        wal_mode = hot_standby >>>        recovery_connections = on > >> That would be a bad idea - there's a significant performance penalty >> from setting wal_le

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Kevin Grittner
Simon Riggs wrote: > Which is what everybody has requested. You continue to use the term "everybody" rather loosely. I don't begrudge people features they want, even when *I* don't need them; but please don't take my lack of argument against adding this feature as a silent request for it. S

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 11:48 -0500, Kevin Grittner wrote: > Simon Riggs wrote: > > > There is no big use case for "run with HS turned off". > > There was mention of running the live database and a warm standby, > with a connection pooler pointed to both so that it could > automatically reconnec

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Tom Lane
Robert Haas writes: > Nobody is proposing otherwise. What Simon and I are proposing is that > if the master is configured to support HS, it comes up on the slave by > default without requiring additional configuration. Now maybe that's > too much spooky action at a distance, but I suspect it IS

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-04-29 at 11:37 -0400, Aidan Van Dyk wrote: >> It's all about the path of least *suprise*. My "least suprise" would >> have been that a similar config I have now with my PITR slaves would >> pretty much still work, and not suddenly start accepting connections, >> a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Kevin Grittner
Simon Riggs wrote: > There is no big use case for "run with HS turned off". There was mention of running the live database and a warm standby, with a connection pooler pointed to both so that it could automatically reconnect on failover. > Everybody wants it on, as long as it works and don't

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 12:17 PM, Aidan Van Dyk wrote: > * Simon Riggs [100429 12:06]: > >> Repeating the same thing when its been refuted doesn't help. What you >> say has not been proposed. >> >> If there is a case for HS-off-by-default, make it. If you want to change >> code, arguing directly

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Aidan Van Dyk
* Simon Riggs [100429 12:14]: HS is the most requested feature across multiple polls and *not* being > able to connect to it is likely to come as a huge surprise to many > people. HS is just as secure as the main database. > > There is no big use case for "run with HS turned off". Everybody want

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-04-29 at 18:40 +0300, Heikki Linnakangas wrote: > >> It's error-prone to control what happens in the standby from the master. >> That's the "action at a distance" effect I mentioned earlier. The master >> should be configured in the master, and each standby should

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Aidan Van Dyk
* Simon Riggs [100429 12:06]: > Repeating the same thing when its been refuted doesn't help. What you > say has not been proposed. > > If there is a case for HS-off-by-default, make it. If you want to change > code, arguing directly against your own position, mentioned many times, > we need a r

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 11:37 -0400, Aidan Van Dyk wrote: > * Simon Riggs [100429 11:24]: > > > > What is the reason for the *extra* "off" switch? Why two? Why "off" > > twice? > > Because I run a PITR slave off a WAL archive... And I'm going to > continue to try and use this setup... I expect to

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 18:40 +0300, Heikki Linnakangas wrote: > It's error-prone to control what happens in the standby from the master. > That's the "action at a distance" effect I mentioned earlier. The master > should be configured in the master, and each standby should configured > in the stand

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-04-29 at 09:48 -0400, Tom Lane wrote: >> Simon Riggs writes: >>> recovery_connections was on by default and unanimous agreement until >>> recently and I don't want to change that now, >> Uh, it was on by default only because a lot of us hadn't noticed that. >> I a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Aidan Van Dyk
* Simon Riggs [100429 11:24]: > > What is the reason for the *extra* "off" switch? Why two? Why "off" > twice? Because I run a PITR slave off a WAL archive... And I'm going to continue to try and use this setup... I expect to try and experiment with wal-streaming too when everything is up to 9,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 09:48 -0400, Tom Lane wrote: > Simon Riggs writes: > > recovery_connections was on by default and unanimous agreement until > > recently and I don't want to change that now, > > Uh, it was on by default only because a lot of us hadn't noticed that. > I agree with Heikki's po

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Tom Lane
Simon Riggs writes: > recovery_connections was on by default and unanimous agreement until > recently and I don't want to change that now, Uh, it was on by default only because a lot of us hadn't noticed that. I agree with Heikki's position here: it should not be on by default.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Dimitri Fontaine
Robert Haas writes: > On Thu, Apr 29, 2010 at 7:19 AM, Heikki Linnakangas > wrote: >> If you're setting up a reporting server, and hot standby can't start, >> the server is not functioning properly. I would like to get an error in >> that case. > > Presumably you will actually try connecting to i

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 9:07 AM, Heikki Linnakangas wrote: > Robert Haas wrote: >> How about this one?  The administrator sets up a master and a slave. >> She's heard about this new Hot Standby feature and so decides to >> enable it on the slave just to play around with it.  Subsequently, she >> t

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 9:00 AM, Heikki Linnakangas wrote: > Robert Haas wrote: >> On Thu, Apr 29, 2010 at 7:19 AM, Heikki Linnakangas >> wrote: >>> Robert Haas wrote: I kind of agree with Simon on this one, except I would probably choose to have just on and off and make on work like hi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Robert Haas wrote: > How about this one? The administrator sets up a master and a slave. > She's heard about this new Hot Standby feature and so decides to > enable it on the slave just to play around with it. Subsequently, she > takes a better job at another company and they hire a new > adminis

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Robert Haas wrote: > On Thu, Apr 29, 2010 at 7:19 AM, Heikki Linnakangas > wrote: >> Robert Haas wrote: >>> I kind of agree with Simon on this one, except I would probably choose >>> to have just on and off and make on work like his auto. >>> >>> In other words, recovery_connections=on means, give

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 7:55 AM, Heikki Linnakangas wrote: > What happened, I don't find that story very compelling because there are an infinite number of ways to have high-availability not work if you start by supposing that the person who sets them up doesn't test them properly and verify that

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > recovery_connections was on by default and unanimous agreement until > recently and I don't want to change that now, just because a change > somewhere else appears to be forcing that but need not be so. We never really had that discussion, until now. It has always been 'on' by

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 14:55 +0300, Heikki Linnakangas wrote: > I'd like to scaremonger Seems so. recovery_connections was on by default and unanimous agreement until recently and I don't want to change that now, just because a change somewhere else appears to be forcing that but need not be so.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-04-29 at 13:46 +0300, Heikki Linnakangas wrote: >> warm standby for high availability - no >> offloading taking filesystem-level backups from master - no > > These can be explicitly turned off. > You're presuming there is benefit in turning recovery_connections =

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 7:19 AM, Heikki Linnakangas wrote: > Robert Haas wrote: >> I kind of agree with Simon on this one, except I would probably choose >> to have just on and off and make on work like his auto. >> >> In other words, recovery_connections=on means, give me recovery >> connections

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Robert Haas wrote: > I kind of agree with Simon on this one, except I would probably choose > to have just on and off and make on work like his auto. > > In other words, recovery_connections=on means, give me recovery > connections if possible, otherwise don't worry about it. If you're setting up

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 13:46 +0300, Heikki Linnakangas wrote: > purpose of the standby - do you want hot standby or not? > reporting - yes > offloading queries from master - yes > offloading pg_dump from master - yes These would work using the proposed default. > warm standby for high availabili

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 5:55 AM, Heikki Linnakangas wrote: >> The most sensible way is to make the default act intelligently depending >> upon what it finds in the control file. If WAL contains hot_standby >> info, then recovery_connections should be on by default, though can be >> turned off expl

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Robert Haas
On Thu, Apr 29, 2010 at 5:38 AM, Heikki Linnakangas wrote: > NOTICE:  WAL archiving is not enabled, you must ensure that all required > WAL segments are streamed or copied through other means to restore the > backup I might think about dropping the words "through other means" from this sentence.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > The whole point of > wal_level discussion was to avoid needing multiple switches to turn > something on. No, the point of wal_level was to make the behavior easier to understand, by uncoupling the level of WAL-logging from various other switches, controling it directly and exp

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 12:55 +0300, Heikki Linnakangas wrote: > Seems overly complicated. If you turn wal_level = hot_standby in the master, then the slaves work, unless you say otherwise. What is complicated about that? > It would be simpler to just set it 'off' by > default. As a parameter, y

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-04-29 at 10:55 +0300, Heikki Linnakangas wrote: > >> Should we change the default to recovery_connections=off ? It is a quite >> big change in default behavior over previous releases that hot standby >> is enabled by default, so maybe that would be the right thing

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Tom Lane wrote: > Yeah. ISTM the real bottom line here is that we have only a weak grasp > on how these features will end up being used; or for that matter what > the common error scenarios will be. I think that for the time being > we should err on the side of being permissive. We can tighten t

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Simon Riggs
On Thu, 2010-04-29 at 10:55 +0300, Heikki Linnakangas wrote: > Should we change the default to recovery_connections=off ? It is a quite > big change in default behavior over previous releases that hot standby > is enabled by default, so maybe that would be the right thing to do anyway. > > Or may

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-29 Thread Heikki Linnakangas
Fujii Masao wrote: > On Thu, Apr 29, 2010 at 1:14 AM, Heikki Linnakangas > wrote: >> Ok, I've finally committed the patch, using wal_level as the name of the >> GUC. > >> !if (InArchiveRecovery && XLogRequestRecoveryConnections) >> !{ >> !if (ControlFile->wal_level < WAL_LEVEL

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Simon Riggs
On Wed, 2010-04-28 at 22:17 +0200, Dimitri Fontaine wrote: > IMO the real fun begins when we talk about multi-slaves support and > their roles (a failover slave wants the master to wait for it to have > applied the WAL before to commit, a reporting slave not so much). So > you'd set the Availabili

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Fujii Masao
On Thu, Apr 29, 2010 at 1:14 AM, Heikki Linnakangas wrote: > Ok, I've finally committed the patch, using wal_level as the name of the > GUC. > ! if (InArchiveRecovery && XLogRequestRecoveryConnections) > ! { > ! if (ControlFile->wal_level < WAL_LEVEL_HOT_STANDBY) > !

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Greg Smith
Dimitri Fontaine wrote: IMO the real fun begins when we talk about multi-slaves support and their roles (a failover slave wants the master to wait for it to have applied the WAL before to commit, a reporting slave not so much). So you'd set the Availability level on each slave and wouldn't commit

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Greg Smith
Aidan Van Dyk wrote: I remember a presentation at pgcon a while ago, it was probaly Fujii (from NTT?) about their log streaming, and at that time, they talked about different "sync" options... It's all outlined at http://wiki.postgresql.org/wiki/Streaming_Replication#Synchronization_capability

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Aidan Van Dyk
* Kevin Grittner [100428 15:51]: > > I don't personally care about streaming replication replaying WAL > > as it comes, or running queries in recovery... > > I'm with you that far, but I wouldn't want the sender to wait for > remote persistence. I remember a presentation at pgcon a while ago

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Dimitri Fontaine
"Kevin Grittner" writes: > Aidan Van Dyk wrote: > >> I'm hoping to be able to build a tool that: >> >> 1) Connects to PG walsender (a la walreceiver) >> 2) Streams WAL from pg master >> 3) Saves WAL into "files" (a la archive)... >> >> i.e. I'm looking to keep a more-up-to-date PITR archive t

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Kevin Grittner
Aidan Van Dyk wrote: > I'm hoping to be able to build a tool that: > > 1) Connects to PG walsender (a la walreceiver) > 2) Streams WAL from pg master > 3) Saves WAL into "files" (a la archive)... > > i.e. I'm looking to keep a more-up-to-date PITR archive than > waiting for traditional WAL fil

Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct

2010-04-28 Thread Tom Lane
Heikki Linnakangas writes: > Hmm, you could start streaming the WAL before you start the backup, so > the fact that you've already removed some segments that are needed to > restore from the backup by the time pg_stop_backup() is called doesn't > necessarily mean that the backup is useless. > You

  1   2   >