Re: [HACKERS] WAL low watermark during base backup

2011-09-09 Thread Magnus Hagander
On Tue, Sep 6, 2011 at 22:35, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Sep 5, 2011 at 11:38 AM, Magnus Hagander mag...@hagander.net wrote: On Sun, Sep 4, 2011 at 19:02, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander mag...@hagander.net wrote:

Re: [HACKERS] WAL low watermark during base backup

2011-09-09 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: If you must have this then make pg_basebackup copy xlog files regularly during the backup. That way your backup can take forever and your primary disk won't fill up. In many cases it actually will take forever, but at least we don't take down the

Re: [HACKERS] WAL low watermark during base backup

2011-09-09 Thread Magnus Hagander
On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Magnus Hagander mag...@hagander.net writes: If you must have this then make pg_basebackup copy xlog files regularly during the backup. That way your backup can take forever and your primary disk won't fill up. In many

Re: [HACKERS] WAL low watermark during base backup

2011-09-09 Thread Florian Pflug
On Sep9, 2011, at 13:48 , Magnus Hagander wrote: On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Magnus Hagander mag...@hagander.net writes: If you must have this then make pg_basebackup copy xlog files regularly during the backup. That way your backup can take

Re: [HACKERS] WAL low watermark during base backup

2011-09-09 Thread Dimitri Fontaine
Florian Pflug f...@phlo.org writes: Couldn't we send all available WAL after each single data-file instead of waiting for all data files to be transferred before sending WAL? +1 (or maybe not at the file boundary but rather driven by archive command with some internal hooking, as the backend

Re: [HACKERS] WAL low watermark during base backup

2011-09-09 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I'm not getting why we need the later one when we have this older one? One of them is for the simple case. It requires a single connection to the server, and it supports

Re: [HACKERS] WAL low watermark during base backup

2011-09-06 Thread Simon Riggs
On Mon, Sep 5, 2011 at 11:38 AM, Magnus Hagander mag...@hagander.net wrote: On Sun, Sep 4, 2011 at 19:02, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander mag...@hagander.net wrote: Attached patch implements a low watermark wal location in the walsender

Re: [HACKERS] WAL low watermark during base backup

2011-09-05 Thread Magnus Hagander
On Sun, Sep 4, 2011 at 19:02, Simon Riggs si...@2ndquadrant.com wrote: On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander mag...@hagander.net wrote: Attached patch implements a low watermark wal location in the walsender shmem array. Setting this value in a walsender prevents transaction log

Re: [HACKERS] WAL low watermark during base backup

2011-09-04 Thread Simon Riggs
On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander mag...@hagander.net wrote: Attached patch implements a low watermark wal location in the walsender shmem array. Setting this value in a walsender prevents transaction log removal prior to this point - similar to how wal_keep_segments work,

Re: [HACKERS] WAL low watermark during base backup

2011-09-03 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: Attached patch implements a low watermark wal location in the walsender shmem array. Setting this value in a walsender prevents transaction log removal prior to this point - similar to how wal_keep_segments work, except with an absolute number

[HACKERS] WAL low watermark during base backup

2011-09-02 Thread Magnus Hagander
Attached patch implements a low watermark wal location in the walsender shmem array. Setting this value in a walsender prevents transaction log removal prior to this point - similar to how wal_keep_segments work, except with an absolute number rather than relative. For now, this is set when

Re: [HACKERS] WAL low watermark during base backup

2011-09-02 Thread Jaime Casanova
On Fri, Sep 2, 2011 at 12:52 PM, Magnus Hagander mag...@hagander.net wrote: Attached patch implements a low watermark wal location in the walsender shmem array. Setting this value in a walsender prevents transaction log removal prior to this point - similar to how wal_keep_segments work,

Re: [HACKERS] WAL low watermark during base backup

2011-09-02 Thread Magnus Hagander
On Fri, Sep 2, 2011 at 20:12, Jaime Casanova ja...@2ndquadrant.com wrote: On Fri, Sep 2, 2011 at 12:52 PM, Magnus Hagander mag...@hagander.net wrote: Attached patch implements a low watermark wal location in the walsender shmem array. Setting this value in a walsender prevents transaction log