Re: [HACKERS] Changing pg_dump default file format

2013-11-19 Thread Robert Haas
On Mon, Nov 18, 2013 at 3:21 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Nov 7, 2013 at 02:40:17PM -0500, Peter Eisentraut wrote: On 11/7/13, 1:00 PM, Josh Berkus wrote: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than

Re: [HACKERS] Changing pg_dump default file format

2013-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: However, pggit log 9f0ae0c82060e3dcd1fa7ac8bbe35a3f9a44dbba does not show that line being added by the diff. I dunno what your pggit script does, but git log doesn't normally show the diff at all. git show 9f0ae0c82060e3dcd1fa7ac8bbe35a3f9a44dbba,

Re: [HACKERS] Changing pg_dump default file format

2013-11-19 Thread Bruce Momjian
On Tue, Nov 19, 2013 at 11:26:34AM -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: However, pggit log 9f0ae0c82060e3dcd1fa7ac8bbe35a3f9a44dbba does not show that line being added by the diff. I dunno what your pggit script does, but git log doesn't normally show the

Re: [HACKERS] Changing pg_dump default file format

2013-11-18 Thread Bruce Momjian
On Thu, Nov 7, 2013 at 02:40:17PM -0500, Peter Eisentraut wrote: On 11/7/13, 1:00 PM, Josh Berkus wrote: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. Note the following

Re: [HACKERS] Changing pg_dump default file format

2013-11-08 Thread Harold Giménez
I don't want to hijack this thread any further, but Craig, thanks for your insight. -Harold On Thu, Nov 7, 2013 at 8:35 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 11/08/2013 11:41 AM, Harold Giménez wrote: On Thu, Nov 7, 2013 at 7:01 PM, Craig Ringer cr...@2ndquadrant.com

Re: [HACKERS] Changing pg_dump default file format

2013-11-08 Thread Josh Berkus
On 11/07/2013 08:35 PM, Craig Ringer wrote: I've seen all sorts of combinations of client and server versions in the wild. Ancient JDBC, ODBC, etc drivers are also common. There's a security compromise out there for the 8.1 JDBC driver which was discovered last year, and folks *keep* reporting

[HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
Hello, For as long as I can remember I have heard complaints about the fact that pg_dump doesn't use a backup format that is supported by the pg_restore program. At every single talk that I give that talks about pg_dump, the people in the majority of the audience agrees that this is rather

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Magnus Hagander
On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: Hello, For as long as I can remember I have heard complaints about the fact that pg_dump doesn't use a backup format that is supported by the pg_restore program. At every single talk that I give that talks about

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Andrew Dunstan
On 11/07/2013 11:02 AM, Joshua D. Drake wrote: Hello, For as long as I can remember I have heard complaints about the fact that pg_dump doesn't use a backup format that is supported by the pg_restore program. At every single talk that I give that talks about pg_dump, the people in the

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Kevin Grittner
Joshua D. Drake j...@commandprompt.com wrote: For as long as I can remember I have heard complaints about the fact that pg_dump doesn't use a backup format that is supported by the pg_restore program. At every single talk that I give that talks about pg_dump, the people in the majority of the

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from -Fp to -Fc. I'm a bit worried about it

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 08:17 AM, Andrew Dunstan wrote: At least it gives you a nice message now instead of the rather cryptic message it used to give: [andrew@emma inst.5715]$ bin/pg_dump -f foo.sql [andrew@emma inst.5715]$ bin/pg_restore foo.sql pg_restore: [archiver] input file appears

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 08:21 AM, Kevin Grittner wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from -Fp to -Fc. -1 in the case that output goes to stdout. It might make sense for when -f is specified. As a DBA, 90%

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Harold Giménez
On Thu, Nov 7, 2013 at 8:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: I humbly request on behalf of those who manage production postgresql instances that we change the

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 08:26 AM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Magnus Hagander
On Thu, Nov 7, 2013 at 5:53 PM, Joshua D. Drake j...@commandprompt.com wrote: On 11/07/2013 08:21 AM, Kevin Grittner wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from -Fp to -Fc. -1 in the case that

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 09:25 AM, Magnus Hagander wrote: PITR is wonderful, I use it all the time. It has exactly zero to do with what we are talking about. We are talking about pg_dump and its usefulness. Are you suggesting that we tell people to not use pg_dump and instead use PITR? For backups,

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Josh Berkus
On 11/07/2013 08:26 AM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 10:00 AM, Josh Berkus wrote: On 11/07/2013 08:26 AM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: I humbly request on behalf of those who manage production postgresql instances that we

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Andres Freund
On 2013-11-07 10:00:53 -0800, Josh Berkus wrote: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. Provided somebody is actually willing to put in the not inconsiderable amount of

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Magnus Hagander
On Thu, Nov 7, 2013 at 7:08 PM, Joshua D. Drake j...@commandprompt.com wrote: On 11/07/2013 10:00 AM, Josh Berkus wrote: On 11/07/2013 08:26 AM, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Nov 7, 2013 at 5:02 PM, Joshua D. Drake j...@commandprompt.com wrote: I

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. That might work. However, just to be contrary: what about the functionality of pg_dumpall?

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Jeff Janes
On Thu, Nov 7, 2013 at 8:53 AM, Joshua D. Drake j...@commandprompt.comwrote: On 11/07/2013 08:21 AM, Kevin Grittner wrote: I humbly request on behalf of those who manage production postgresql instances that we change the default backup file format from -Fp to -Fc. -1 in the case that

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread David Fetter
On Thu, Nov 07, 2013 at 01:45:20PM -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. That might work. However,

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Kevin Grittner
Joshua D. Drake j...@commandprompt.com wrote: PITR is wonderful, I use it all the time. It has exactly zero to do with what we are talking about. We are talking about pg_dump and its usefulness. Are you suggesting that we tell people to not use pg_dump and instead use PITR? Yes.  (Well,

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Alvaro Herrera
Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. That might work. However, just to be contrary: what about the

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Jan Lentfer
Am 07.11.2013 19:08, schrieb Joshua D. Drake: On 11/07/2013 10:00 AM, Josh Berkus wrote: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. I am not opposed to that. Allow pg_dump

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 11:01 AM, Alvaro Herrera wrote: Maybe we could provide a master controller program (pg_backup or whatever name we agree on), which could receive commands much like pg_ctl. $ pg_backup --help pg_backup is a backup handler program for PostgreSQL. Usage: pg_backup backup [-p

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Peter Eisentraut
On 11/7/13, 1:00 PM, Josh Berkus wrote: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. Note the following code in pg_dump.c: /* Set default options based on progname */

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 10:54 AM, Kevin Grittner wrote: That is who I am thinking of. A DBA team may have hundreds of databases to manage, each with many scripts which have been running nicely for years. A change like this is bound to break some of those crontab scripts they may not even remember they

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Andrew Dunstan
On 11/07/2013 02:27 PM, Jan Lentfer wrote: Btw, how hard would it be, to have pg_restore and now also pg_dump run with -j option do some ordering of work by size of e.g. the tables? E.g. if you run with -j4 it would make sense to start working on the largest tables (and it's indexes) first

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Josh Berkus
All, I'm amused at how this has become a let's pile on everything which has ever been missing in pg_dump into one thread. Agree with Tom that if we're going to create a new program name, we should fix the pg_dumpall issue as well. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Joshua D. Drake
On 11/07/2013 12:42 PM, Josh Berkus wrote: All, I'm amused at how this has become a let's pile on everything which has ever been missing in pg_dump into one thread. Well it has been broken longer than most of our utilities. Sorry... not broken but certainly not complete. It is to be

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Andrew Dunstan
On 11/07/2013 06:33 PM, Joshua D. Drake wrote: On 11/07/2013 12:42 PM, Josh Berkus wrote: All, I'm amused at how this has become a let's pile on everything which has ever been missing in pg_dump into one thread. Well it has been broken longer than most of our utilities. Sorry... not

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Craig Ringer
On 11/08/2013 12:53 AM, Harold Giménez wrote: Just a thought: If it's the right thing to do, why not do it in small steps? For example, couldn't we deprecate the behavior on 9.4, and switch over in 9.5? By deprecate I mean a warning message on STDERR making it clear they're relying on

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Craig Ringer
On 11/08/2013 12:55 AM, Joshua D. Drake wrote: We have certainly done worse in the past. 8.3 comes to mind. I am trying to consider the future here, as we continue to grow. Why leave thorns for the sake of leaving thorns? 8.3 was _bad_. It's been a major upgrade barrier, and a huge pain. I

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Craig Ringer
On 11/08/2013 07:33 AM, Joshua D. Drake wrote: All I want to start is this simple fix. I don't know who is going to step up to work on pg_dump/all considering it is the bastard step child of the project. We only work on it when we absolutely have to (and CMD is just as guilty as the rest).

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Harold Giménez
On Thu, Nov 7, 2013 at 7:01 PM, Craig Ringer cr...@2ndquadrant.com wrote: (a) Lots of people only upgrade every two, three, or even more major versions. I'm dealing with clients on 8.3, and people still pop up on Stack Overflow with 8.1 sometimes! These people don't ever see the deprecated

Re: [HACKERS] Changing pg_dump default file format

2013-11-07 Thread Craig Ringer
On 11/08/2013 11:41 AM, Harold Giménez wrote: On Thu, Nov 7, 2013 at 7:01 PM, Craig Ringer cr...@2ndquadrant.com mailto:cr...@2ndquadrant.com wrote: (a) Lots of people only upgrade every two, three, or even more major versions. I'm dealing with clients on 8.3, and people