Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Zeugswetter Andreas ADI SD
A nice improvement on that would be to have a rearchive_command to allow to sync the new bytes written since a previous archive_command (so it needs a new placeholder start from this byte). This allows writing dd seek=%s skip=%s count=%b bs=1 But after a log switch nothing is filling that

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Alvaro Herrera
Zeugswetter Andreas ADI SD wrote: The probably useful next step would be to pass the current length to the archive_command, so it can write the filled part of the file without the need for a filter. I can see that helping a lot, but not by writing onto the file on disk. If

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Tom Lane
Zeugswetter Andreas ADI SD [EMAIL PROTECTED] writes: I am suggesting to add such a placeholder for the size of the filled part of the log. The archiver has not got that information, and can't compute it any faster than the called command could. regards, tom lane

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: Not quite as good. Since the archiver process can't actually deliver this number in a lightweight manner, all it goes to show is that the filter code compares reasonably well in performance with dd and cat. I'd definitely vote for leaving it as a

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Zeugswetter Andreas ADI SD
The probably useful next step would be to pass the current length to the archive_command, so it can write the filled part of the file without the need for a filter. I can see that helping a lot, but not by writing onto the file on disk. If the file is nearly empty, that would be a

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Kevin Grittner
On Fri, Sep 28, 2007 at 9:38 AM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: On Fri, Sep 28, 2007 at 5:53 AM, in message [EMAIL PROTECTED], Zeugswetter Andreas ADI SD [EMAIL PROTECTED] wrote: archive_command=dd if=%p of=/backup/WAL/%f bs=1 count=%b I think

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-28 Thread Kevin Grittner
On Fri, Sep 28, 2007 at 5:53 AM, in message [EMAIL PROTECTED], Zeugswetter Andreas ADI SD [EMAIL PROTECTED] wrote: I think you misunderstood what I meant. The actual archive command is constructed by expanding certain placeholders. I am suggesting to add such a placeholder for the size of

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-27 Thread Heikki Linnakangas
Kevin Grittner wrote: [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: We should also document that this is designed to help compress files that aren't full because we switched early because of archive_timeout. Attached is a modified version to implement both of these. I also

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-27 Thread Simon Riggs
On Thu, 2007-09-27 at 08:31 +0100, Heikki Linnakangas wrote: Kevin Grittner wrote: [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: We should also document that this is designed to help compress files that aren't full because we switched early because of archive_timeout.

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-27 Thread Zeugswetter Andreas ADI SD
Attached is a modified version to implement both of these. I also bailed out if there was surplus input. I tried an optimization of allocating a separate buffer for outputting the zeros, to avoid repeated memset calls. It didn't seem to make a very big difference; do you think it's

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-27 Thread Simon Riggs
On Thu, 2007-09-27 at 10:17 +0200, Zeugswetter Andreas ADI SD wrote: Attached is a modified version to implement both of these. I also bailed out if there was surplus input. I tried an optimization of allocating a separate buffer for outputting the zeros, to avoid repeated memset

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-27 Thread Kevin Grittner
On Thu, Sep 27, 2007 at 3:17 AM, in message [EMAIL PROTECTED], Zeugswetter Andreas ADI SD [EMAIL PROTECTED] wrote: The probably useful next step would be to pass the current length to the archive_command, so it can write the filled part of the file without the need for a filter. I can

Re: [HACKERS] [FEATURE REQUEST] Streaming Onlinebackup (MaybeOFFTOPIC)

2007-09-08 Thread Gregory Stark
Hannu Krosing [EMAIL PROTECTED] writes: Is this apoc9009 guy real ? Please, just don't respond. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ?