Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2009-01-27 Thread Tom Lane
Heikki Linnakangas writes: > (replying to a very old message, since I just bumped into this in review) > Alvaro Herrera wrote: >> The new message is not translatable, the original was. > Doesn't really matter since it's an elog(), not ereport(). ... which is wrong in itself, since it's certainly

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2009-01-27 Thread Heikki Linnakangas
(replying to a very old message, since I just bumped into this in review) Alvaro Herrera wrote: Simon Riggs wrote: On Fri, 2008-09-12 at 14:14 -0400, Alvaro Herrera wrote: Simon Riggs wrote: --- 5716,5725 CheckpointStats.ckpt_sync_end_t,

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Simon Riggs
On Tue, 2008-10-07 at 18:19 +0300, Heikki Linnakangas wrote: > The bottom line is that hot standby is a big feature, and probably a big > patch. No amount of version control will work around that. Finishing all > that in a few weeks is a very ambitious goal. I wish you luck, and I > wish I cou

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Heikki Linnakangas
Alvaro Herrera wrote: Simon Riggs wrote: I'm just grumpy because I can't see a way to do the patch-on-patch-on-patch that I'll need to make this all work for Nov 1. So big patch here we come. But that's just the way it is and I'll stop honking about it. This is one of the problems that DVCSs a

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Dave Page
On Tue, Oct 7, 2008 at 4:08 PM, Simon Riggs <[EMAIL PROTECTED]> wrote: > > Not convinced this is the right time to invest in side activities, but > if you think so, I'll look into it. > > Anybody wanting to write or link to a Simon's Guide, most welcome. Heikki will be presenting a talk about GIT

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: >>> I'm just grumpy because I can't see a way to do the >>> patch-on-patch-on-patch that I'll need to make this all work for Nov 1. >>> So big patch here we come. But that's just the way it is and I'll stop >>> honking about it. >> >> This is one of the pr

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Simon Riggs
On Tue, 2008-10-07 at 10:37 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > I'm just grumpy because I can't see a way to do the > > patch-on-patch-on-patch that I'll need to make this all work for Nov 1. > > So big patch here we come. But that's just the way it is and I'll stop > > honkin

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Robert Haas
>> I'm just grumpy because I can't see a way to do the >> patch-on-patch-on-patch that I'll need to make this all work for Nov 1. >> So big patch here we come. But that's just the way it is and I'll stop >> honking about it. > > This is one of the problems that DVCSs are supposed to solve ... have

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Alvaro Herrera
Simon Riggs wrote: > I'm just grumpy because I can't see a way to do the > patch-on-patch-on-patch that I'll need to make this all work for Nov 1. > So big patch here we come. But that's just the way it is and I'll stop > honking about it. This is one of the problems that DVCSs are supposed to so

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Simon Riggs
On Tue, 2008-10-07 at 10:08 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > A patch specifically marked as "required for other work" has been > > delayed by more than 5 weeks on queue and nobody was ever assigned to > > review it. That was exactly the problem CommitFests were

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > A patch specifically marked as "required for other work" has been > delayed by more than 5 weeks on queue and nobody was ever assigned to > review it. That was exactly the problem CommitFests were supposed to > resolve and from my perspective this is a syst

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-07 Thread Simon Riggs
On Thu, 2008-10-02 at 19:07 -0400, Andrew Dunstan wrote: > Simon Riggs wrote: > > Just seen this patch has been bounced into November CommitFest, even > > though the new patch fixes all of the concerns raised. > > > > I'm concerned that this is going to make the final Hot Standby patch > > fairl

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-02 Thread Andrew Dunstan
Simon Riggs wrote: On Thu, 2008-09-25 at 18:28 -0400, Tom Lane wrote: Simon Riggs <[EMAIL PROTECTED]> writes: Version 7 After reading this for awhile, I realized that there is a rather fundamental problem with it: it switches into "consistent recovery" mode as soon as it's rea

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-10-02 Thread Simon Riggs
On Thu, 2008-09-25 at 18:28 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Version 7 > > After reading this for awhile, I realized that there is a rather > fundamental problem with it: it switches into "consistent recovery" > mode as soon as it's read WAL beyond ControlFile-

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Mon, 2008-09-29 at 11:24 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote: > >> ... If we crash and restart, we'll have to get to the end > >> of this file before we start letting backends in; which might be further > >> than

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote: >> ... If we crash and restart, we'll have to get to the end >> of this file before we start letting backends in; which might be further >> than we actually got before the crash, but not too much further be

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I think we can get away with writing the LSN value to disk, as you > > suggested, but only every so often. No need to do it after every WAL > > record, just consistently every so often, so it gives us

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I think we can get away with writing the LSN value to disk, as you > suggested, but only every so often. No need to do it after every WAL > record, just consistently every so often, so it gives us a point at > which we know we are safe. Huh? How does that

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Mon, 2008-09-29 at 08:46 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > ... That kinda works, but the problem is that restartpoints are time based, > > not log based. We need them to be deterministic for us to rely upon them > > in the above way. > > Right, but the perfor

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > ... That kinda works, but the problem is that restartpoints are time based, > not log based. We need them to be deterministic for us to rely upon them > in the above way. Right, but the performance disadvantages of making them strictly log-distance-based a

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-29 Thread Simon Riggs
On Sun, 2008-09-28 at 21:16 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > >> It does nothing AFAICS for the > >> problem that when restarting archive recovery from a restartpoint, > >> it's not clear when it is safe to start letting in backends. You need > >> to get past the

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-28 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: >> It does nothing AFAICS for the >> problem that when restarting archive recovery from a restartpoint, >> it's not clear when it is safe to start letting in backends. You need >> to get past the highest LSN that has made it out to disk, and there is >> no g

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-28 Thread Simon Riggs
On Sun, 2008-09-28 at 14:02 -0400, Tom Lane wrote: > It does nothing AFAICS for the > problem that when restarting archive recovery from a restartpoint, > it's not clear when it is safe to start letting in backends. You need > to get past the highest LSN that has made it out to disk, and there i

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-28 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Thu, 2008-09-25 at 18:28 -0400, Tom Lane wrote: >> After reading this for awhile, I realized that there is a rather >> fundamental problem with it: it switches into "consistent recovery" >> mode as soon as it's read WAL beyond ControlFile->minRecoveryPoi

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-26 Thread Simon Riggs
On Fri, 2008-09-26 at 11:20 +0100, Simon Riggs wrote: > > After reading this for awhile, I realized that there is a rather > > fundamental problem with it: it switches into "consistent recovery" > > mode as soon as it's read WAL beyond ControlFile->minRecoveryPoint. > > In a crash recovery situat

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-26 Thread Simon Riggs
On Thu, 2008-09-25 at 18:28 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Version 7 > Anyway, that's sufficiently bad that I'm bouncing the patch for > reconsideration. No problem, I understand this needs discussion. There's less detail here than first appears. There a

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Version 7 After reading this for awhile, I realized that there is a rather fundamental problem with it: it switches into "consistent recovery" mode as soon as it's read WAL beyond ControlFile->minRecoveryPoint. In a crash recovery situation that typically

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-23 Thread Simon Riggs
On Mon, 2008-09-22 at 23:06 +0100, Simon Riggs wrote: > On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > > >> Do we really need a checkpoint there at all? > > > > > "Timelines only change at s

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-22 Thread Simon Riggs
On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > >> Do we really need a checkpoint there at all? > > > "Timelines only change at shutdown checkpoints". > > Hmm. I *think* that that is just a deb

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > >> Do we really need a checkpoint there at all? > > > "Timelines only change at shutdown checkpoints". > > Hmm. I *think* that that is just a deb

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: >> Do we really need a checkpoint there at all? > "Timelines only change at shutdown checkpoints". Hmm. I *think* that that is just a debugging crosscheck rather than a critical property. But yeah, it w

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Having some trouble trying to get a clean state change from recovery to > > normal mode. > > > Startup needs to be able to write WAL at the end of recovery so it can > > write a ShutdownCheckpoint, ye

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Having some trouble trying to get a clean state change from recovery to > normal mode. > Startup needs to be able to write WAL at the end of recovery so it can > write a ShutdownCheckpoint, yet must not be allowed to write WAL before > that. Other services

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Thu, 2008-09-18 at 09:05 +0100, Simon Riggs wrote: > Feels like I should shutdown the bgwriter after recovery and then > allow it to be cranked up again after normal processing starts, and do > all of this through postmaster state changes. That way bgwriter > doesn't need to do a dynamic state

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-18 Thread Simon Riggs
On Tue, 2008-09-16 at 15:45 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> wrote: > > Testing takes a while on this, I probably won't complete it until > > Friday. So enclosed patch is for eyeballs only at this stage. > > What's the status on that patch? Having some trouble trying to g