Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: More verbosely (not numbered because they're not a sequence or progression) - if no recovery.conf is present we do crash recovery to end of logs on pg_control timeline (i.e. current) Check. - if recovery.conf is present and we do not specify a target

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Simon Riggs
On Wed, 2004-07-21 at 23:42, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: More verbosely (not numbered because they're not a sequence or progression) - if no recovery.conf is present we do crash recovery to end of logs on pg_control timeline (i.e. current) Check. - if

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: PITR should cope with these scenarios. These are described reasonably closely but not as exact mechanical tests, so as to ensure that if multiple solutions exist to these recovery scenarios that all paths are tested. [ snip ... ] Now *my* head is

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Christopher Kings-Lynne
That gives us enough to talk through and begin some testing. Anybody have any other horror stories, bring 'em on. I think that the PITR docs will have to be written in two sections. One will need to be a pure reference that orthogonally describes the options, etc. The other section will need

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-20 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Anybody see any holes in this design? God help the DBA who deletes a history file with needed information. Or edits it inappropriately. Why can't every log file contain a header that says which timeline it's part of and which timeline the preceding log file

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-20 Thread Simon Riggs
On Tue, 2004-07-20 at 00:58, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: where the default is notarget and if you specify a target, the default target_in_timeline is latest. I think actually the default target has to be the timeline ID found in pg_control -- otherwise you get

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 04:31, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: The way you write this makes me think you might mean you would allow: we can start recovering in one timelines, then rollforward takes us through all the timeline nexus points required to get us to the target

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Some further thinking from that base... Perhaps timelines should be nest-numbered: (using 0 as a counter also) 0 etc is the original branch 0.1 is the first recovery off the original branch 0.2 is the second recovery off the original branch 0.1.1 is the

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
I wrote: I think there's really no way around the issue: somehow we've got to keep some meta-history outside the $PGDATA area, if we want to do this in a clean fashion. After further thought I think we can fix this stuff by creating a history file for each timeline. This will make recovery

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 16:58, Tom Lane wrote: I think there's really no way around the issue: somehow we've got to keep some meta-history outside the $PGDATA area, if we want to do this in a clean fashion. We could perhaps expect the archive area to store it, but I'm a bit worried about the

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 19:33, Tom Lane wrote: I wrote: I think there's really no way around the issue: somehow we've got to keep some meta-history outside the $PGDATA area, if we want to do this in a clean fashion. After further thought I think we can fix this stuff by creating a history

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: The crucial time is when re-running recoveries repeatedly and if we write the manual with sufficient red ink then we'll avoid this. But heck, not having your history file is only as bad as not having added timelines in the first place. Not great, just more

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2004-07-19 at 19:33, Tom Lane wrote: When doing WAL archiving a history file can be copied off to the archive area by the existing archiver mechanism (ie, we'll make a .ready file for it as soon as it's written). Need to check the archive code

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I think we're heatedly agreeing again. Yeah, I think so. I'll get started on this tomorrow. where the default is notarget and if you specify a target, the default target_in_timeline is latest. I think actually the default target has to be the timeline

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 23:15, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2004-07-19 at 19:33, Tom Lane wrote: * When we need to do recovery, we first identify the source timeline (either by reading the current timeline ID from pg_control, or the DBA can tell us with a

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-18 Thread Simon Riggs
On Sat, 2004-07-17 at 21:36, Tom Lane wrote: If we do not add timeline numbers to WAL file names, we will be forced to destroy information during recovery. Consider the following scenario: 1. You have a WAL directory containing, say, WAL segments 0010 to 0020 (for the purposes of this

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-18 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: The way you write this makes me think you might mean you would allow: we can start recovering in one timelines, then rollforward takes us through all the timeline nexus points required to get us to the target timeline. Sure. Let's draw a diagram:

[HACKERS] Why we really need timelines *now* in PITR

2004-07-17 Thread Tom Lane
If we do not add timeline numbers to WAL file names, we will be forced to destroy information during recovery. Consider the following scenario: 1. You have a WAL directory containing, say, WAL segments 0010 to 0020 (for the purposes of this example I won't bother typing out realistic 16-digit