Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Simon Riggs
Bruce Momjian... Alvaro Herrera wrote: Anyway I think we are way past feature freeze, and XLog compression could made it into 8.1 without loss of functionality. At that point we will say now PITR uses less space on the archiver and that's it. (Maybe we could even think about

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Zeugswetter Andreas SB SD
PS: but something you *could* do in 8.0 is replace cp by gzip to archive compressed files that way. How about replacing the page image records with a same size dummy record that only contains a dummy header and all 0's. If the archiver cares about space he will use some sort of compression

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Zeugswetter Andreas SB SD
Re-thinking the whole purpose of the additional full page images appended to the xlog records, I now understand and agree with Tom's comment in the docs that we don't need to include those additional full page images for PITR - they only exist to correctly recover the database in the event of

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-24 Thread Tom Lane
Zeugswetter Andreas SB SD [EMAIL PROTECTED] writes: But it can contain newer blocks than WAL records would expect. Will it not matter if e.g. a page split for a btree index is already in the index file, but a WAL record exists, that references the not yet split page? No. This is not different

[HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Simon Riggs
One of the possible barriers to adoption of PITR is the volume of the logs themselves. Maybe this isn't a problem for now, maybe it is. Re-thinking the whole purpose of the additional full page images appended to the xlog records, I now understand and agree with Tom's comment in the docs that we

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Alvaro Herrera
On Mon, Aug 23, 2004 at 10:03:26PM +0100, Simon Riggs wrote: Simon, I raise this now because I'm thinking that this functionality really ought to be in the main production 8.0 release. Not sure if anybody will agreebut that's what I'm thinking now, based upon what seems like a simple

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Bruce Momjian
Alvaro Herrera wrote: On Mon, Aug 23, 2004 at 10:03:26PM +0100, Simon Riggs wrote: Simon, I raise this now because I'm thinking that this functionality really ought to be in the main production 8.0 release. Not sure if anybody will agreebut that's what I'm thinking now, based upon

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Gaetano Mendola
Simon Riggs wrote: As a result, I have thought that there may be a way to remove those pages from the xlog files immediately before being copied away to archive, without effecting crash recovery logic AT ALL. The archiver process could read the xlog files and re-write them exactly as read to

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: As a result, I have thought that there may be a way to remove those pages from the xlog files immediately before being copied away to archive, without effecting crash recovery logic AT ALL. This isn't all that easy. The main problem I can see is that you

Re: [HACKERS] PITR: XLog File compression on Archive

2004-08-23 Thread Alvaro Herrera
On Tue, Aug 24, 2004 at 12:05:33AM +0200, Gaetano Mendola wrote: Simon Riggs wrote: As a result, I have thought that there may be a way to remove those pages from the xlog files immediately before being copied away to archive, without effecting crash recovery logic AT ALL. So this means