Re: pg_dump --with-* options

2025-07-30 Thread Álvaro Herrera
On 2025-Jul-30, Jeff Davis wrote: > On Wed, 2025-07-30 at 10:23 +0200, Álvaro Herrera wrote: > > Maybe we should invent a new > > switch, something like > >   --include=[schema,data,statistics] > > with which users can give one or more comma-separated types to be > > included in the dump.  > > R

Re: pg_dump --with-* options

2025-07-30 Thread Jeff Davis
On Wed, 2025-07-30 at 10:23 +0200, Álvaro Herrera wrote: > Maybe we should invent a new > switch, something like >   --include=[schema,data,statistics] > with which users can give one or more comma-separated types to be > included in the dump.  Robert Treat brought up a similar idea before: http

Re: pg_dump --with-* options

2025-07-30 Thread Álvaro Herrera
On 2025-Jun-25, Fujii Masao wrote: > For the record, my vote is: default "off" for pg_dump and pg_dumpall, > and "on" for pg_restore. I don't know if this horse is already dead, so bear with me while I beat it a little more. > [...] we could simplify further: > > * Keep: --schema-only, --da

Re: pg_dump --with-* options

2025-07-29 Thread Jeff Davis
On Tue, 2025-07-29 at 20:22 +0200, Álvaro Herrera wrote: > Please move the switches themselves out of the translatable message, > otherwise there are too many of them.  For instance, Thank you for looking, v2 attached. Regards, Jeff Davis From 61b0239f17a1c7220de32699e95c6b365accbb88 Mon

Re: pg_dump --with-* options

2025-07-29 Thread Jeff Davis
On Wed, 2025-06-18 at 10:21 -0700, Jeff Davis wrote: > On Wed, 2025-06-18 at 10:43 -0500, Nathan Bossart wrote: > > IIUC the current proposal is to: > > > > * Dump/restore stats by default. We don't have a consensus for that, so unless a few people make an abrupt turnaround, this will remain off

Re: pg_dump --with-* options

2025-07-29 Thread Álvaro Herrera
On 2025-Jul-29, Jeff Davis wrote: > + /* reject conflicting "only-" and "with-" options */ > + if (data_only && with_schema) > + pg_fatal("options -a/--data-only and --with-schema cannot be > used together"); > + if (data_only && with_statistics) > + pg_fatal("

Re: pg_dump --with-* options

2025-07-29 Thread Jeff Davis
On Thu, 2025-07-10 at 10:42 -0700, Jeff Davis wrote: > On Wed, 2025-06-18 at 10:21 -0700, Jeff Davis wrote: > >   * reject the combination of an "only" option and a "with" option > > There seems to be a rough consensus on this point. Patch attached. Regards, Jeff Davis From f668a980ca73

Re: pg_dump --with-* options

2025-07-29 Thread Jeff Davis
On Fri, 2025-07-11 at 09:12 +0900, Fujii Masao wrote: > But to do that, we'd either need to make pg_dump dump statistics > by default, or allow redundant options like --statistics in > pg_restore, > even though it already restores statistics by default. Redundant options might be annoying, but I d

Re: pg_dump --with-* options

2025-07-10 Thread Fujii Masao
On 2025/07/11 2:57, Jeff Davis wrote: On Wed, 2025-06-25 at 08:18 +0900, Fujii Masao wrote: For pg_dump and pg_dumpall, I agree with Jeff's idea in [1], but if the statistics is skipped by default, I don't think we need a --no-statistics option. So, here's how I think the options should work:

Re: pg_dump --with-* options

2025-07-10 Thread Jeff Davis
On Wed, 2025-06-25 at 08:18 +0900, Fujii Masao wrote: > For pg_dump and pg_dumpall, I agree with Jeff's idea in [1], > but if the statistics is skipped by default, I don't think > we need a --no-statistics option. So, here's how I think > the options should work: > > * Keep: --schema-only, --

Re: pg_dump --with-* options

2025-07-10 Thread Jeff Davis
On Wed, 2025-06-18 at 10:21 -0700, Jeff Davis wrote: >   * reject the combination of an "only" option and a "with" option There seems to be a rough consensus on this point. Should we move ahead with this small change and see if we can get consensus to go further? Regards, Jeff Davis

Re: pg_dump --with-* options

2025-06-30 Thread Jeff Davis
On Mon, 2025-06-23 at 13:38 -0400, Robert Haas wrote: > What confuses me about what you've written here specifically is that > pg_dump and pg_restore are different programs with different option > sets. So when you say we need both --with-statistics and > --no-statistics, I guess that's true, but

Re: pg_dump --with-* options

2025-06-25 Thread Greg Sabino Mullane
On Wed, Jun 25, 2025 at 10:36 AM Nathan Bossart wrote: > > This is so close to ideal. It's just that the first bullet should be > "off by default" :) > > If we did that, the only way to dump statistics would be > --statistics-only, right? You wouldn't be able to include statistics along > with o

Re: pg_dump --with-* options

2025-06-25 Thread Nathan Bossart
On Wed, Jun 25, 2025 at 08:18:28AM +0900, Fujii Masao wrote: > For pg_dump and pg_dumpall, I agree with Jeff's idea in [1], > but if the statistics is skipped by default, I don't think > we need a --no-statistics option. So, here's how I think > the options should work: > > * Keep: --schema-on

Re: pg_dump --with-* options

2025-06-25 Thread Nathan Bossart
On Tue, Jun 24, 2025 at 06:14:55PM -0400, Greg Sabino Mullane wrote: > On Wed, Jun 18, 2025 at 11:43 AM Nathan Bossart > wrote: >> IIUC the current proposal is to: >> >> * Dump/restore stats by default. >> * Keep the --no-statistics, --no-schema, and --no-data options. >> * Keep the --statistics-o

Re: pg_dump --with-* options

2025-06-24 Thread Fujii Masao
On 2025/06/25 5:07, Robert Haas wrote: On Tue, Jun 24, 2025 at 12:48 PM Nathan Bossart wrote: On Mon, Jun 23, 2025 at 01:38:10PM -0400, Robert Haas wrote: I had thought we had a consensus that pg_upgrade should preserve stats but regularly pg_dump shouldn't include them; perhaps I misunders

Re: pg_dump --with-* options

2025-06-24 Thread Nathan Bossart
On Mon, Jun 23, 2025 at 01:38:10PM -0400, Robert Haas wrote: > I had thought we had a consensus that pg_upgrade should preserve stats > but regularly pg_dump shouldn't include them; perhaps I misunderstood > or that changed. I think it's a bit of both. I skimmed through the past discussions and f

Re: pg_dump --with-* options

2025-06-24 Thread Robert Haas
On Tue, Jun 24, 2025 at 12:48 PM Nathan Bossart wrote: > On Mon, Jun 23, 2025 at 01:38:10PM -0400, Robert Haas wrote: > > I had thought we had a consensus that pg_upgrade should preserve stats > > but regularly pg_dump shouldn't include them; perhaps I misunderstood > > or that changed. > > I thin

Re: pg_dump --with-* options

2025-06-24 Thread Greg Sabino Mullane
On Wed, Jun 18, 2025 at 11:43 AM Nathan Bossart wrote: > IIUC the current proposal is to: > > * Dump/restore stats by default. > * Keep the --no-statistics, --no-schema, and --no-data options. > * Keep the --statistics-only, --schema-only, and --data-only options. > * Remove the --with-statistics

Re: pg_dump --with-* options

2025-06-23 Thread Robert Haas
On Wed, Jun 18, 2025 at 1:21 PM Jeff Davis wrote: > The only downside of this approach is that we'd be stuck with both -- > with-statistics and --no-statistics forever. That's a bit inconsistent > with the other options, and it doesn't satisfy Robert's concern about > the --help output. But Robert

Re: pg_dump --with-* options

2025-06-18 Thread Jeff Davis
On Wed, 2025-06-18 at 10:43 -0500, Nathan Bossart wrote: > IIUC the current proposal is to: > > * Dump/restore stats by default. > * Keep the --no-statistics, --no-schema, and --no-data options. > * Keep the --statistics-only, --schema-only, and --data-only options. > * Remove the --with-statistic

Re: pg_dump --with-* options

2025-06-18 Thread Nathan Bossart
On Wed, Jun 18, 2025 at 09:53:01AM -0700, Jeff Davis wrote: > On Wed, 2025-06-18 at 10:43 -0500, Nathan Bossart wrote: >> IIUC the current proposal is to: >> >> * Dump/restore stats by default. > > IIUC some people still object to this. Turning stats off by default was > on the Open Items list. A

Re: pg_dump --with-* options

2025-06-18 Thread Jeff Davis
On Wed, 2025-06-18 at 10:43 -0500, Nathan Bossart wrote: > IIUC the current proposal is to: > > * Dump/restore stats by default. IIUC some people still object to this. Turning stats off by default was on the Open Items list. At this point I think we need a pretty strong consensus to override that

Re: pg_dump --with-* options

2025-06-18 Thread Nathan Bossart
On Wed, Jun 18, 2025 at 08:29:16AM -0700, Jeff Davis wrote: > Actually, I take that back, we can't just remove --no-statistics. > Remember that statistics currently default to "on" for pg_restore even > though they default "off" for pg_dump. > > So pg_restore still needs a way to turn stats off.

Re: pg_dump --with-* options

2025-06-18 Thread Jeff Davis
On Thu, 2025-06-12 at 08:58 -0700, Jeff Davis wrote: > On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: > > If the idea is to remove all options for default behavior, we'd be > > removing > > --no-statistics, --with-data, and --with-schema at this point. > > That's OK with me. Actually, I

Re: pg_dump --with-* options

2025-06-17 Thread Jeff Davis
On Mon, 2025-06-16 at 15:35 -0400, Corey Huinker wrote: > > I think this is the exact sort of confusion caused by having two of > the three types default to on in all circumstances, and one default > to off in one special circumstance. That's certainly a part of the confusion, but the "--x-only"

Re: pg_dump --with-* options

2025-06-16 Thread Fujii Masao
On 2025/06/17 9:58, Nathan Bossart wrote: On Mon, Jun 16, 2025 at 07:09:17PM -0400, Tom Lane wrote: I find myself increasingly persuaded by Corey's point of view ... +1 Can you clarify how using on-by-default would simplify things? I'm not sure it actually makes the options simpler. Rega

Re: pg_dump --with-* options

2025-06-16 Thread Fujii Masao
On 2025/06/17 4:35, Corey Huinker wrote: I noticed that --statistics (i.e., the current --with-statistics) causes statistics to be dumped even when used with --data-only or --schema-only. So, as far as I understand, here are the possible combinations of dump targets and options

Re: pg_dump --with-* options

2025-06-16 Thread Nathan Bossart
On Mon, Jun 16, 2025 at 07:09:17PM -0400, Tom Lane wrote: > I find myself increasingly persuaded by Corey's point of view ... +1 -- nathan

Re: pg_dump --with-* options

2025-06-16 Thread Tom Lane
Jeff Davis writes: > Does it make any sense to be off by default in 18 and on in some later > release? Probably not, especially if part of the argument for on-by-default is to allow simplification of the switch set. We don't get that benefit if we ship with off-by-default, and we won't be able t

Re: pg_dump --with-* options

2025-06-16 Thread Jeff Davis
On Mon, 2025-06-16 at 16:09 -0500, Nathan Bossart wrote: > So perhaps there's not as strong of a > consensus as we thought.  Maybe we should ask for any new/updated > votes. Does it make any sense to be off by default in 18 and on in some later release? Regards Jeff Davis

Re: pg_dump --with-* options

2025-06-16 Thread Nathan Bossart
On Mon, Jun 16, 2025 at 03:35:48PM -0400, Corey Huinker wrote: > I think this is the exact sort of confusion caused by having two of the > three types default to on in all circumstances, and one default to off in > one special circumstance. I revisited the main thread to see how folks voted. Ther

Re: pg_dump --with-* options

2025-06-16 Thread Corey Huinker
> > I noticed that --statistics (i.e., the current --with-statistics) causes > statistics to be dumped even when used with --data-only or --schema-only. > So, as far as I understand, here are the possible combinations of dump > targets and options: > Those should also be mutually exclusive, and I'

Re: pg_dump --with-* options

2025-06-14 Thread Fujii Masao
On 2025/06/14 5:32, Nathan Bossart wrote: On Fri, Jun 13, 2025 at 08:58:04AM -0700, Jeff Davis wrote: On Fri, 2025-06-13 at 09:39 +0900, Fujii Masao wrote: By the way, if we keep --with-statistics in pg_dump, are we planning to continue using the --with-xxx naming pattern for new options tha

Re: pg_dump --with-* options

2025-06-13 Thread Corey Huinker
> > > Good point. Now that we are getting rid of some of the other options, > we don't need to worry about consistency with them, and I think we > should just use "--statistics". The point of the --with flags was to future proof commands to preserve behavior in case the defaults ever changed. Th

Re: pg_dump --with-* options

2025-06-13 Thread Corey Huinker
> > One of the challenges in the current case is that it is not obvious how > --with-data, --no-data, --data-only etc. are connected. If that were > clearer, then the way these options should combine or conflict would > hopefully follow somewhat naturally. > They all should be mutually exclusive,

Re: pg_dump --with-* options

2025-06-13 Thread Nathan Bossart
On Fri, Jun 13, 2025 at 08:58:04AM -0700, Jeff Davis wrote: > On Fri, 2025-06-13 at 09:39 +0900, Fujii Masao wrote: >> By the way, if we keep --with-statistics in pg_dump, are we planning >> to >> continue using the --with-xxx naming pattern for new options that >> specify extra data to dump? > >

Re: pg_dump --with-* options

2025-06-13 Thread Jeff Davis
On Fri, 2025-06-13 at 09:39 +0900, Fujii Masao wrote: > By the way, if we keep --with-statistics in pg_dump, are we planning > to > continue using the --with-xxx naming pattern for new options that > specify extra data to dump? Good point. Now that we are getting rid of some of the other options,

Re: pg_dump --with-* options

2025-06-13 Thread Jeff Davis
On Fri, 2025-06-13 at 07:22 +0200, Peter Eisentraut wrote: > > I don't think it's simple to start using "last option wins" > > behavior > > now ... > It makes sense to raise an error if the specified options cannot be > consolidated in an obvious way. To me, "last option wins" means that you don'

Re: pg_dump --with-* options

2025-06-13 Thread Daniel Gustafsson
> On 13 Jun 2025, at 02:39, Fujii Masao wrote: > By the way, if we keep --with-statistics in pg_dump, are we planning to > continue using the --with-xxx naming pattern for new options that > specify extra data to dump? I just wondered because pg_dump already has > other naming styles like --seque

Re: pg_dump --with-* options

2025-06-12 Thread Peter Eisentraut
On 12.06.25 23:20, Jeff Davis wrote: On Thu, 2025-06-12 at 21:16 +0200, Peter Eisentraut wrote: Do we have other options that are order-sensitive? I think most of them are.  For example: psql -p 5432 -p 5433 initdb --data-checksums --no-data-checksums postgres --shared-buffers=1GB --shared-bu

Re: pg_dump --with-* options

2025-06-12 Thread Greg Sabino Mullane
On Thu, Jun 12, 2025 at 4:12 PM Corey Huinker wrote: (peacefully skimming thread...) ... > If we're hot to remove options, how about we remove the sections flags? > Their utility is reliant upon the user understanding exactly which things > go in which section, and further assumes that everything

Re: pg_dump --with-* options

2025-06-12 Thread Fujii Masao
On 2025/06/12 23:52, Nathan Bossart wrote: On Thu, Jun 12, 2025 at 10:18:56AM -0400, Robert Haas wrote: On Fri, Jun 6, 2025 at 11:40 AM Nathan Bossart wrote: On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: What is the purpose of the --with-data option? Dumping the data i

Re: pg_dump --with-* options

2025-06-12 Thread Fujii Masao
On 2025/06/13 6:12, Jeff Davis wrote: On Thu, 2025-06-12 at 15:57 -0500, Nathan Bossart wrote: FWIW I don't have a tremendously strong opinion about --statistics- only. Same here. I won't cast a vote on this particular issue, as long as the functionality is available. I prefer keeping it

Re: pg_dump --with-* options

2025-06-12 Thread Jeff Davis
On Thu, 2025-06-12 at 21:16 +0200, Peter Eisentraut wrote: > > Do we have other options that are order-sensitive? > > I think most of them are.  For example: > > psql -p 5432 -p 5433 > initdb --data-checksums --no-data-checksums > postgres --shared-buffers=1GB --shared-buffers=2GB Interesting. I

Re: pg_dump --with-* options

2025-06-12 Thread Jeff Davis
On Thu, 2025-06-12 at 15:57 -0500, Nathan Bossart wrote: > FWIW I don't have a tremendously strong opinion about --statistics- > only. Same here. I won't cast a vote on this particular issue, as long as the functionality is available. Regards, Jeff Davis

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 04:39:00PM -0400, Corey Huinker wrote: > On Thu, Jun 12, 2025 at 4:22 PM Nathan Bossart > wrote: >> I do think this is useful functionality, I only suggested removing it >> because AFAICT it is redundant, i.e., you can accomplish the same thing >> with --with-statistics --n

Re: pg_dump --with-* options

2025-06-12 Thread Corey Huinker
On Thu, Jun 12, 2025 at 4:22 PM Nathan Bossart wrote: > On Thu, Jun 12, 2025 at 04:12:35PM -0400, Corey Huinker wrote: > > The use case for --statistics-only is to extract the existing statistics > > for the tables and indexes that are involved in a given query that is > > giving you problems, al

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 04:12:35PM -0400, Corey Huinker wrote: > The use case for --statistics-only is to extract the existing statistics > for the tables and indexes that are involved in a given query that is > giving you problems, allowing you to apply those statistics to an existing > QA/dev dat

Re: pg_dump --with-* options

2025-06-12 Thread Corey Huinker
On Thu, Jun 12, 2025 at 1:36 PM Robert Haas wrote: > On Thu, Jun 12, 2025 at 11:58 AM Jeff Davis wrote: > > On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: > > > If the idea is to remove all options for default behavior, we'd be > > > removing > > > --no-statistics, --with-data, and --w

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 10:07:05PM +0200, Laurenz Albe wrote: > I must be missing something, but I think --no-statistics is sorely needed. > How else can I get the effect of > > pg_dump --no-statistics mydb This was recently changed to be the default behavior (see commit 34eb2a8). -- nathan

Re: pg_dump --with-* options

2025-06-12 Thread Laurenz Albe
On Thu, 2025-06-12 at 13:36 -0400, Robert Haas wrote: > On Thu, Jun 12, 2025 at 11:58 AM Jeff Davis wrote: > > On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: > > > If the idea is to remove all options for default behavior, we'd be > > > removing > > > --no-statistics, --with-data, and --

Re: pg_dump --with-* options

2025-06-12 Thread Peter Eisentraut
On 12.06.25 17:14, Jeff Davis wrote: On Thu, 2025-06-12 at 15:47 +0200, Peter Eisentraut wrote: My initial guess was that --with-data can override --no-data.  That would have been pretty standard "last option wins" behavior.  But pg_dump rejects that.  Personally, I think that is kind of wrong.

Re: pg_dump --with-* options

2025-06-12 Thread Robert Haas
On Thu, Jun 12, 2025 at 11:58 AM Jeff Davis wrote: > On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: > > If the idea is to remove all options for default behavior, we'd be > > removing > > --no-statistics, --with-data, and --with-schema at this point. > > That's OK with me. Same. > >

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 08:58:15AM -0700, Jeff Davis wrote: > On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: >> If the idea is to remove all options for default behavior, we'd be >> removing >> --no-statistics, --with-data, and --with-schema at this point. > > That's OK with me. > >>  

Re: pg_dump --with-* options

2025-06-12 Thread Jeff Davis
On Thu, 2025-06-12 at 09:52 -0500, Nathan Bossart wrote: > If the idea is to remove all options for default behavior, we'd be > removing > --no-statistics, --with-data, and --with-schema at this point. That's OK with me. >   Maybe we > could go a step further and even rip out --statistics-only (i

Re: pg_dump --with-* options

2025-06-12 Thread Jeff Davis
On Thu, 2025-06-12 at 10:18 -0400, Robert Haas wrote: > Am I too late to propose ripping this out? As long as we keep the functionality, I'm fine changing the options/names around at this point. Regards, Jeff Davis

Re: pg_dump --with-* options

2025-06-12 Thread Jeff Davis
On Thu, 2025-06-12 at 15:47 +0200, Peter Eisentraut wrote: > My initial guess was that --with-data can override --no-data.  That > would have been pretty standard "last option wins" behavior.  But > pg_dump rejects that.  Personally, I think that is kind of wrong. Do we have other options that a

Re: pg_dump --with-* options

2025-06-12 Thread Nathan Bossart
On Thu, Jun 12, 2025 at 10:18:56AM -0400, Robert Haas wrote: > On Fri, Jun 6, 2025 at 11:40 AM Nathan Bossart > wrote: >> On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: >> > What is the purpose of the --with-data option? Dumping the data is the >> > default. Is this to overri

Re: pg_dump --with-* options

2025-06-12 Thread Fujii Masao
On 2025/06/12 22:47, Peter Eisentraut wrote: On 06.06.25 17:39, Nathan Bossart wrote: On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: We have    -a, --data-only  dump only the data, not the schema or statistics    --no-data    do not dump data    --with-data   

Re: pg_dump --with-* options

2025-06-12 Thread Robert Haas
On Fri, Jun 6, 2025 at 11:40 AM Nathan Bossart wrote: > On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: > > We have > > > > -a, --data-only dump only the data, not the schema or statistics > > --no-datado not dump data > > --with-data dump the data

Re: pg_dump --with-* options

2025-06-12 Thread Peter Eisentraut
On 06.06.25 17:39, Nathan Bossart wrote: On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: We have -a, --data-only dump only the data, not the schema or statistics --no-datado not dump data --with-data dump the data # this one is new (and the

Re: pg_dump --with-* options

2025-06-06 Thread Nathan Bossart
On Fri, Jun 06, 2025 at 09:14:32AM +0200, Peter Eisentraut wrote: > We have > > -a, --data-only dump only the data, not the schema or statistics > --no-datado not dump data > --with-data dump the data # this one is new > > (and there is also --section=data), and t

pg_dump --with-* options

2025-06-06 Thread Peter Eisentraut
I'm looking at the new in PG18 pg_dump --with-* options, and I'm having trouble understanding them. (I did not look into the source code or the git or mailing list history for this, to try to understand it as a user.) We have -a, --data-only dump only the data, not the