[HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-17 Thread Tom Lane
I have noticed that a large fraction of the I/O done by 7.1 is associated with initializing new segments of the WAL log for use. (We have to physically fill each segment with zeroes to ensure that the system has actually allocated a whole 16MB to it; otherwise we fall victim to the "hole-saving" a

AW: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Zeugswetter Andreas SB
> I just had an idea about how to avoid this cost: > why not recycle old log segments? At the point where the code > currently deletes a no-longer-needed segment, just rename it to > become the next created-in-advance segment. Yes, since I already suggested this on Feb 26. I naturally think thi

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Patrick Macdonald
Hmmm... my prior appends to this newsgroup are stalled. Hopefully, they'll be available soon. Tom Lane wrote: > > What you may really be saying is that the existing scheme for management > of log segments is inappropriate for PIT usage; if so feel free to > propose a better one. But I don't se

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Tom Lane
Patrick Macdonald <[EMAIL PROTECTED]> writes: > Yes, but in a very roundabout way (or so it seems). The main point > that I was trying to illustrate was that if a database supports > point-in-time recovery, recycling of the only available log segments > is a bad thing. Certainly, but deleting

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Bruce Momjian
> Nonetheless, at some point an old WAL segment will become deletable > (unless you have infinite space on your WAL disk). ISTM that at that > point, it makes sense to consider recycling the file rather than > deleting it. Of course, if you plan to keep your WAL files on the same drive, you don'

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-17 Thread Tom Lane
Patrick Macdonald <[EMAIL PROTECTED]> writes: > I understand your solution is for the existing architecture which does > not support point-in-time recovery. If this item is picked up, your > solution will become a stumbling block due the above mentioned log > extent deletions. Hmm, I don't see w

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Bruce Momjian
> Hmmm... my prior appends to this newsgroup are stalled. Hopefully, > they'll be available soon. > > Tom Lane wrote: > > > > What you may really be saying is that the existing scheme for management > > of log segments is inappropriate for PIT usage; if so feel free to > > propose a better one.

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-17 Thread Tom Lane
Patrick Macdonald <[EMAIL PROTECTED]> writes: > Well, notion and actual practice can be mutually exclusive. Your > initial message stated that you would like to rename the log segment. > This insinuated that the log segment was not moved. Therefore, a > straight rename would cause problems with

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-17 Thread Bruce Momjian
> I have noticed that a large fraction of the I/O done by 7.1 is > associated with initializing new segments of the WAL log for use. > (We have to physically fill each segment with zeroes to ensure that > the system has actually allocated a whole 16MB to it; otherwise we > fall victim to the "hole

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-23 Thread Patrick Macdonald
Tom Lane wrote: > > Patrick Macdonald <[EMAIL PROTECTED]> writes: > > I understand your solution is for the existing architecture which does > > not support point-in-time recovery. If this item is picked up, your > > solution will become a stumbling block due the above mentioned log > > extent d

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-23 Thread Patrick Macdonald
Tom, What you are describing is a pseudo circular log. Other database systems (such as DB2) support the concept of both circular and recoverable logs. Recoverable is named this way because recoverable logs can be used in point-in-time recovery. Both methods support crash recovery. In genera

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Patrick Macdonald
Bruce Momjian wrote: > > > Hmmm... my prior appends to this newsgroup are stalled. Hopefully, > > they'll be available soon. > > > > Tom Lane wrote: > > > > > > What you may really be saying is that the existing scheme for management > > > of log segments is inappropriate for PIT usage; if so fe

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Larry Rosenman
CE on a table). LER >>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<< On 7/18/01, 11:35:04 AM, Bruce Momjian <[EMAIL PROTECTED]> wrote regarding Re: [HACKERS] Idea: recycle

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Bruce Momjian
> > > Yes, but in a very roundabout way (or so it seems). The main point > > > that I was trying to illustrate was that if a database supports > > > point-in-time recovery, recycling of the only available log segments > > > is a bad thing. And, yes, in practice if you have point-in-time > > > re

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Bruce Momjian
> > > Of course not. As mentioned, you'd probably archive them with your > > > backup(s). > > > > You mean the nigthly backup? Why not do a pg_dump and be done with it. > > But the purpose of point-in-time recovery is to restore your backup > and then use the WAL to bring the backed up image

Re: [HACKERS] Idea: recycle WAL segments, don't delete/recreate 'em

2001-07-18 Thread Patrick Macdonald
Bruce Momjian wrote: > > > > > Yes, but in a very roundabout way (or so it seems). The main point > > > > that I was trying to illustrate was that if a database supports > > > > point-in-time recovery, recycling of the only available log segments > > > > is a bad thing. And, yes, in practice if

Re: AW: [HACKERS] Idea: recycle WAL segments, don't delete/recreate ' em

2001-07-18 Thread Tom Lane
Zeugswetter Andreas SB <[EMAIL PROTECTED]> writes: > Yes, since I already suggested this on Feb 26. So you did. Darn, I thought it was original ;-) regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our ex