Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-12 Thread Abhijit Menon-Sen
At 2015-06-11 14:38:03 +0900, langote_amit...@lab.ntt.co.jp wrote: > > > On the other hand, I don't like the idea of doing (3) by adding > > command line arguments to pg_basebackup and adding a new option to > > the command. I don't think that level of "flexibility" is justified; > > it would also

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Alvaro Herrera
Michael Paquier wrote: > After spending the night thinking about that, honestly, I think that > we should go with (2) and keep the base backup as light-weight as > possible and not bother about a GUC. (3) would need some extra > intelligence to decide if some files can be skipped or not. Imagine >

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Magnus Hagander
On Jun 11, 2015 7:38 AM, "Amit Langote" wrote: > > On 2015-06-11 PM 02:20, Abhijit Menon-Sen wrote: > > At 2015-06-10 13:22:27 -0400, robertmh...@gmail.com wrote: > >> > >> (1) include pg_log in pg_basebackup as we do currently > >> (2) exclude it > >> (3) add a switch controlling whether or not i

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Michael Paquier
On Thu, Jun 11, 2015 at 2:39 PM, Abhijit Menon-Sen wrote: > At 2015-06-11 14:28:36 +0900, michael.paqu...@gmail.com wrote: >> >> After spending the night thinking about that, honestly, I think that >> we should go with (2) and keep the base backup as light-weight as >> possible and not bother abou

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Amit Langote
On 2015-06-11 PM 02:20, Abhijit Menon-Sen wrote: > At 2015-06-10 13:22:27 -0400, robertmh...@gmail.com wrote: >> >> (1) include pg_log in pg_basebackup as we do currently >> (2) exclude it >> (3) add a switch controlling whether or not it gets excluded >> >> I can live with (3), but I bet most peop

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Abhijit Menon-Sen
At 2015-06-11 14:28:36 +0900, michael.paqu...@gmail.com wrote: > > After spending the night thinking about that, honestly, I think that > we should go with (2) and keep the base backup as light-weight as > possible and not bother about a GUC. OK. Then the patch I posted earlier should be sufficien

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-11 Thread Michael Paquier
On Thu, Jun 11, 2015 at 2:20 PM, Abhijit Menon-Sen wrote: > At 2015-06-10 13:22:27 -0400, robertmh...@gmail.com wrote: >> >> I'm not clear on which of these options you are voting for: >> >> (1) include pg_log in pg_basebackup as we do currently >> (2) exclude it >> (3) add a switch controlling wh

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Abhijit Menon-Sen
At 2015-06-10 13:22:27 -0400, robertmh...@gmail.com wrote: > > I'm not clear on which of these options you are voting for: > > (1) include pg_log in pg_basebackup as we do currently > (2) exclude it > (3) add a switch controlling whether or not it gets excluded > > I can live with (3), but I bet

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Joshua D. Drake
On 06/10/2015 10:22 AM, Robert Haas wrote: On Wed, Jun 10, 2015 at 1:12 PM, Joshua D. Drake wrote: On 06/10/2015 10:01 AM, Andres Freund wrote: On 2015-06-10 09:57:17 -0700, Jeff Janes wrote: Mine goal isn't that. My goal is to have a consistent backup without having to shut down the serve

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Robert Haas
On Wed, Jun 10, 2015 at 1:12 PM, Joshua D. Drake wrote: > On 06/10/2015 10:01 AM, Andres Freund wrote: >> On 2015-06-10 09:57:17 -0700, Jeff Janes wrote: >>> Mine goal isn't that. My goal is to have a consistent backup without >>> having to shut down the server to take a cold one, or having to ma

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Joshua D. Drake
On 06/10/2015 10:01 AM, Andres Freund wrote: On 2015-06-10 09:57:17 -0700, Jeff Janes wrote: Mine goal isn't that. My goal is to have a consistent backup without having to shut down the server to take a cold one, or having to manually juggle the pg_start_backup, etc. commands. A basebackup

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Andres Freund
On 2015-06-10 09:57:17 -0700, Jeff Janes wrote: > Mine goal isn't that. My goal is to have a consistent backup without > having to shut down the server to take a cold one, or having to manually > juggle the pg_start_backup, etc. commands. A basebackup won't necessarily give you a consistent log t

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Jeff Janes
On Wed, Jun 10, 2015 at 8:29 AM, Robert Haas wrote: > On Mon, Jun 8, 2015 at 12:09 AM, Michael Paquier > wrote: > >> Recently, one of our customers has had a basebackup fail because pg_log > >> contained files that were >8GB: > >> FATAL: archive member "pg_log/postgresql-20150119.log" too large

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-10 Thread Robert Haas
On Mon, Jun 8, 2015 at 12:09 AM, Michael Paquier wrote: >> Recently, one of our customers has had a basebackup fail because pg_log >> contained files that were >8GB: >> FATAL: archive member "pg_log/postgresql-20150119.log" too large for tar >> format >> >> I think pg_basebackup should also skip

Re: [HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-07 Thread Michael Paquier
On Mon, Jun 8, 2015 at 12:42 PM, Abhijit Menon-Sen wrote: > This is a followup to a 2014-02 discussion that led to pg_stats_temp > being excluded from pg_basebackup. At the time, it was discussed to > exclude pg_log as well, but nothing eventually came of that. It seems to be that: http://www.pos

[HACKERS] skipping pg_log in basebackup (was Re: pg_basebackup and pg_stat_tmp directory)

2015-06-07 Thread Abhijit Menon-Sen
Hi. This is a followup to a 2014-02 discussion that led to pg_stats_temp being excluded from pg_basebackup. At the time, it was discussed to exclude pg_log as well, but nothing eventually came of that. Recently, one of our customers has had a basebackup fail because pg_log contained files that we