Hello,
pg_dump ignores the dumping of data in foreign tables
on purpose, this patch makes it optional as the user maybe
wants to manage the data in the foreign servers directly from
Postgres. Opinions?
Cheers
Luis M Carril
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backu
Hello,
pg_dump ignores the dumping of data in foreign tables
on purpose, this patch makes it optional as the user maybe
wants to manage the data in the foreign servers directly from
Postgres. Opinions?
Cheers
Luis M Carril
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backu
> On 28 Jun 2019, at 16:49, Luis Carril wrote:
> pg_dump ignores the dumping of data in foreign tables
> on purpose, this patch makes it optional as the user maybe
> wants to manage the data in the foreign servers directly from
> Postgres. Opinions?
Wouldn’t that have the potential to
Hi
pá 28. 6. 2019 v 16:50 odesílatel Luis Carril
napsal:
> Hello,
> pg_dump ignores the dumping of data in foreign tables
> on purpose, this patch makes it optional as the user maybe
> wants to manage the data in the foreign servers directly from
> Postgres. Opinions?
>
It has sense for
pá 28. 6. 2019 v 17:17 odesílatel Daniel Gustafsson
napsal:
> > On 28 Jun 2019, at 16:49, Luis Carril wrote:
>
> > pg_dump ignores the dumping of data in foreign tables
> > on purpose, this patch makes it optional as the user maybe
> > wants to manage the data in the foreign servers direct
Daniel Gustafsson writes:
>> On 28 Jun 2019, at 16:49, Luis Carril wrote:
>> pg_dump ignores the dumping of data in foreign tables
>> on purpose, this patch makes it optional as the user maybe
>> wants to manage the data in the foreign servers directly from
>> Postgres. Opinions?
> Wouldn’t th
pá 28. 6. 2019 v 17:30 odesílatel Tom Lane napsal:
> Daniel Gustafsson writes:
> >> On 28 Jun 2019, at 16:49, Luis Carril wrote:
> >> pg_dump ignores the dumping of data in foreign tables
> >> on purpose, this patch makes it optional as the user maybe
> >> wants to manage the data in the foreig
>Restoring content of FDW table via pg_restore or psql can be dangerous -
>there I see a risk, and can be nice to allow it only >with some form of
>safeguard.
>
>I think so important questions is motivation for dumping FDW - a) clonning
>(has sense for me and it is safe), b) real backup >(req
> On 28 Jun 2019, at 17:30, Tom Lane wrote:
> > Yeah, I think the feature as-proposed is a shotgun that's much more likely
> > to cause problems than solve them. Almost certainly, what people would
> > really need is the ability to dump individual foreign tables' data not
> > everything. (I al
> > On 28 Jun 2019, at 19:55, Luis Carril wrote:
> > What about providing a list of FDW servers instead of an all or nothing
> > option? In that way the user really has to do a conscious decision to dump
> > the content of the foreign tables for > > a specific server, this would
> > allow dist
> On 12 Jul 2019, at 16:08, Luis Carril wrote:
>
> > > On 28 Jun 2019, at 19:55, Luis Carril wrote:
> > > What about providing a list of FDW servers instead of an all or nothing
> > > option? In that way the user really has to do a conscious decision to
> > > dump the content of the foreign ta
On 15.07.19 12:06, Daniel Gustafsson wrote:
On 12 Jul 2019, at 16:08, Luis Carril wrote:
On 28 Jun 2019, at 19:55, Luis Carril wrote:
What about providing a list of FDW servers instead of an all or nothing option?
In that way the user really has to do a conscious decision to dump the content
Hi Luis,
On 1/29/20 11:05 AM, Peter Eisentraut wrote:
On 2020-01-21 10:36, Luis Carril wrote:
Yes we can support --include-foreign-data without parallel option and
later add support for parallel option as a different patch.
Hi,
I've attached a new version of the patch in which an error
I am just responding on the latest mail on this thread. But the question is
about functionality. The proposal is to add a single flag
--include-foreign-data which controls whether or not data is dumped for all
the foreign tables in a database. That may not serve the purpose. A foreign
table may poi
is Carril ; vignesh C ;
Peter Eisentraut ; Alvaro Herrera
; Daniel Gustafsson ; Laurenz Albe
; PostgreSQL Hackers
Subject: Re: Option to dump foreign data in pg_dump
I am just responding on the latest mail on this thread. But the question is
about functionality. The proposal is to add a single
Hi Luis,
Please don't top post. Also be careful to quote prior text when
replying. Your message was pretty hard to work through -- i.e. figuring
out what you said vs. what you were replying to.
On 3/5/20 8:51 AM, Luis Carril wrote:
At this point I would like to leave the patch as is, and d
v8 attached.
I modified Luis' v7 a little bit by putting the ftserver acquisition in
the main pg_class query instead of adding one separate query for each
foreign table. That seems better overall.
I don't understand why this code specifically disallows the empty string
as an option to --dump-for
On 2020-Mar-23, Alvaro Herrera wrote:
> > This seems like an overreaction. The whole point of lockTableForWorker() is
> > to avoid deadlocks, but foreign tables don't have locks, so it's not a
> > problem. I think you can just skip foreign tables in lockTableForWorker()
> > using the same logic
On 2020-Jan-29, Peter Eisentraut wrote:
> On 2020-01-21 10:36, Luis Carril wrote:
> > > Yes we can support --include-foreign-data without parallel option and
> > > later add support for parallel option as a different patch.
> >
> > I've attached a new version of the patch in which an error i
> On 23 Mar 2020, at 21:40, Alvaro Herrera wrote:
> I don't understand why this code specifically disallows the empty string
> as an option to --dump-foreign-data. The other pattern-matching options
> don't do that. This seems to have been added in response to Daniel's
> review[1], but I don't
Hi Luis,
Here is a few comment for me
*I suggest the option to be just –foreign-data because if we make it
–include-foreign-data its expected to have –exclude-foreign-data option
too.
*please add test case
* + if (tdinfo->filtercond || tbinfo->relkind == RELKIND_FOREIGN_TABLE)
filter condition
On Mon, Jul 15, 2019 at 6:09 PM Luis Carril wrote:
>
> On 15.07.19 12:06, Daniel Gustafsson wrote:
>
Few comments:
As you have specified required_argument in above:
+ {"include-foreign-data", required_argument, NULL, 11},
The below check may not be required:
+ if (strcmp(optarg, "") == 0)
+ {
+
On Thu, Sep 19, 2019 at 3:08 PM vignesh C wrote:
>
> On Mon, Jul 15, 2019 at 6:09 PM Luis Carril wrote:
> >
> > On 15.07.19 12:06, Daniel Gustafsson wrote:
> >
> Few comments:
>
> As you have specified required_argument in above:
> + {"include-foreign-data", required_argument, NULL, 11},
>
> The
Hello,
thanks for the comments!
*I suggest the option to be just –foreign-data because if we make it
–include-foreign-data its expected to have –exclude-foreign-data option too.
Several pg_dump options have no counterpart, e.g --enable-row-security does not
have a disable (which is the defa
On Fri, Sep 20, 2019 at 6:20 PM Luis Carril wrote:
> Hello,
>thanks for the comments!
>
> * + if (tdinfo->filtercond || tbinfo->relkind == RELKIND_FOREIGN_TABLE)
>
> filter condition is not implemented completely yet so the logic only work
> on foreign table so I think its better to handle it
On Fri, Sep 20, 2019 at 6:20 PM Luis Carril
mailto:luis.car...@swarm64.com>> wrote:
Hello,
thanks for the comments!
* + if (tdinfo->filtercond || tbinfo->relkind == RELKIND_FOREIGN_TABLE)
filter condition is not implemented completely yet so the logic only work on
foreign table so I think it
On 20 Sep 2019, at 17:20, Luis Carril wrote:I took a look at this patch again today for a review of the latest version.While I still think it's a potential footgun due to read-only FDW's, I can seeusecases for having it so I'm mildly +1 on adding it.The patch applies to master with a little bit of
On Sat, 2019-11-09 at 21:38 +0100, Daniel Gustafsson wrote:
> I took a look at this patch again today for a review of the latest version.
> While I still think it's a potential footgun due to read-only FDW's, I can see
> usecases for having it so I'm mildly +1 on adding it.
I don't feel good about
Hello
a new version of the patch with the tests from Daniel (thanks!) and the
nitpicks.
I don't feel good about this feature.
pg_dump should not dump any data that are not part of the database
being dumped.
If you restore such a dump, the data will be inserted into the foreign table,
right?
> On 12 Nov 2019, at 12:12, Luis Carril wrote:
>a new version of the patch with the tests from Daniel (thanks!) and the
> nitpicks.
The nitpicks have been addressed. However, it seems that the new file
containing the test FDW seems missing from the new version of the patch. Did
you forget
The nitpicks have been addressed. However, it seems that the new file
containing the test FDW seems missing from the new version of the patch. Did
you forget to git add the file?
Yes, I forgot, thanks for noticing. New patch attached again.
Cheers
Luis M Carril
diff --git a/doc/src/sgml/ref
On 2019-Nov-12, Luis Carril wrote:
> But, not all foreign tables are necessarily in a remote server like
> the ones referenced by the postgres_fdw.
> In FDWs like swarm64da, cstore, citus or timescaledb, the foreign
> tables are part of your database, and one could expect that a dump of
> the data
> On 12 Nov 2019, at 15:21, Luis Carril wrote:
>
>> The nitpicks have been addressed. However, it seems that the new file
>> containing the test FDW seems missing from the new version of the patch. Did
>> you forget to git add the file?
> Yes, I forgot, thanks for noticing. New patch attached a
Alvaro Herrera writes:
> On 2019-Nov-12, Luis Carril wrote:
>> But, not all foreign tables are necessarily in a remote server like
>> the ones referenced by the postgres_fdw.
>> In FDWs like swarm64da, cstore, citus or timescaledb, the foreign
>> tables are part of your database, and one could exp
On 2019-Nov-12, Luis Carril wrote:
> The nitpicks have been addressed. However, it seems that the new file
> containing the test FDW seems missing from the new version of the patch. Did
> you forget to git add the file?
>
> Yes, I forgot, thanks for noticing. New patch attached again.
Luis,
I
Alvaro Herrera
Sent: Thursday, November 28, 2019 3:31 PM
To: Luis Carril
Cc: Daniel Gustafsson ; Laurenz Albe
; vignesh C ; PostgreSQL Hackers
Subject: Re: Option to dump foreign data in pg_dump
On 2019-Nov-12, Luis Carril wrote:
> The nitpicks have been addressed. However, it seems
> On 28 Jun 2019, at 16:49, Luis Carril wrote:
> pg_dump ignores the dumping of data in foreign tables
> on purpose, this patch makes it optional as the user maybe
> wants to manage the data in the foreign servers directly from
> Postgres. Opinions?
Wouldn’t that have the potential to
Hi
pá 28. 6. 2019 v 16:50 odesílatel Luis Carril
napsal:
> Hello,
> pg_dump ignores the dumping of data in foreign tables
> on purpose, this patch makes it optional as the user maybe
> wants to manage the data in the foreign servers directly from
> Postgres. Opinions?
>
It has sense for
pá 28. 6. 2019 v 17:17 odesílatel Daniel Gustafsson
napsal:
> > On 28 Jun 2019, at 16:49, Luis Carril wrote:
>
> > pg_dump ignores the dumping of data in foreign tables
> > on purpose, this patch makes it optional as the user maybe
> > wants to manage the data in the foreign servers direct
Daniel Gustafsson writes:
>> On 28 Jun 2019, at 16:49, Luis Carril wrote:
>> pg_dump ignores the dumping of data in foreign tables
>> on purpose, this patch makes it optional as the user maybe
>> wants to manage the data in the foreign servers directly from
>> Postgres. Opinions?
> Wouldn’t th
pá 28. 6. 2019 v 17:30 odesílatel Tom Lane napsal:
> Daniel Gustafsson writes:
> >> On 28 Jun 2019, at 16:49, Luis Carril wrote:
> >> pg_dump ignores the dumping of data in foreign tables
> >> on purpose, this patch makes it optional as the user maybe
> >> wants to manage the data in the foreig
>Restoring content of FDW table via pg_restore or psql can be dangerous -
>there I see a risk, and can be nice to allow it only >with some form of
>safeguard.
>
>I think so important questions is motivation for dumping FDW - a) clonning
>(has sense for me and it is safe), b) real backup >(req
> On 28 Jun 2019, at 17:30, Tom Lane wrote:
> > Yeah, I think the feature as-proposed is a shotgun that's much more likely
> > to cause problems than solve them. Almost certainly, what people would
> > really need is the ability to dump individual foreign tables' data not
> > everything. (I al
> > On 28 Jun 2019, at 19:55, Luis Carril wrote:
> > What about providing a list of FDW servers instead of an all or nothing
> > option? In that way the user really has to do a conscious decision to dump
> > the content of the foreign tables for > > a specific server, this would
> > allow dist
> On 12 Jul 2019, at 16:08, Luis Carril wrote:
>
> > > On 28 Jun 2019, at 19:55, Luis Carril wrote:
> > > What about providing a list of FDW servers instead of an all or nothing
> > > option? In that way the user really has to do a conscious decision to
> > > dump the content of the foreign ta
On 15.07.19 12:06, Daniel Gustafsson wrote:
On 12 Jul 2019, at 16:08, Luis Carril wrote:
On 28 Jun 2019, at 19:55, Luis Carril wrote:
What about providing a list of FDW servers instead of an all or nothing option?
In that way the user really has to do a conscious decision to dump the content
On Fri, Nov 29, 2019 at 2:10 PM Luis Carril wrote:
>
> Luis,
>
> It seems you've got enough support for this concept, so let's move
> forward with this patch. There are some comments from Tom about the
> patch; would you like to send an updated version perhaps?
>
> Thanks
>
> Hi,
>
> I've attach
modify it while is being dumped.
Cheers,
Luis M Carril
From: vignesh C
Sent: Tuesday, January 14, 2020 1:48 AM
To: Luis Carril
Cc: Alvaro Herrera ; Daniel Gustafsson
; Laurenz Albe ; PostgreSQL Hackers
Subject: Re: Option to dump foreign data in pg_dump
On Fri
On Tue, Jan 14, 2020 at 5:22 PM Luis Carril wrote:
> Can you have a look at dump with parallel option. Parallel option will
> take a lock on table while invoking lockTableForWorker. May be this is
> not required for foreign tables.
> Thoughts?
>
> I tried with -j and found no issue. I guess that
Suggestions?
Cheers
Luis M Carril
From: vignesh C
Sent: Thursday, January 16, 2020 10:01 AM
To: Luis Carril
Cc: Alvaro Herrera ; Daniel Gustafsson
; Laurenz Albe ; PostgreSQL Hackers
Subject: Re: Option to dump foreign data in pg_dump
On Tue, Jan 14, 2020 at 5
On Mon, Jan 20, 2020 at 8:34 PM Luis Carril wrote:
>
>
> Hi Vignesh,
>
>yes you are right I could reproduce it also with 'file_fdw'. The issue is
> that LOCK is not supported on foreign tables, so I guess that the safest
> solution is to make the --include-foreign-data incompatible with --jo
Yes we can support --include-foreign-data without parallel option and
later add support for parallel option as a different patch.
Hi,
I've attached a new version of the patch in which an error is emitted if
the parallel backup is used with the --include-foreign-data option.
Cheers
Luis M
On Tue, Jan 21, 2020 at 3:06 PM Luis Carril wrote:
>
> Yes we can support --include-foreign-data without parallel option and
> later add support for parallel option as a different patch.
>
> Hi,
>
> I've attached a new version of the patch in which an error is emitted if
> the parallel backup
Thanks for working on the comments. I noticed one behavior is
different when --table option is specified. When --table is specified
the following are not getting dumped:
CREATE SERVER foreign_server
I felt the above also should be included as part of the dump when
include-foreign-data option is sp
On 2020-01-21 10:36, Luis Carril wrote:
Yes we can support --include-foreign-data without parallel option and
later add support for parallel option as a different patch.
Hi,
I've attached a new version of the patch in which an error is
emitted if the parallel backup is used with the --in
On Wed, Jan 29, 2020 at 2:00 PM Luis Carril wrote:
>
> Thanks for working on the comments. I noticed one behavior is
> different when --table option is specified. When --table is specified
> the following are not getting dumped:
> CREATE SERVER foreign_server
>
> I felt the above also should be in
On 2020-Mar-23, Alvaro Herrera wrote:
> COPY public.ft1 (c1, c2, c3) FROM stdin;
> pg_dump: error: query failed: ERROR: foreign-data wrapper "dummy" has no
> handler
> pg_dump: error: query was: COPY (SELECT c1, c2, c3 FROM public.ft1 ) TO
> stdout;
>
> Maybe what we should do just verify that
On 2020-Mar-24, Alvaro Herrera wrote:
> On 2020-Mar-23, Alvaro Herrera wrote:
>
> > COPY public.ft1 (c1, c2, c3) FROM stdin;
> > pg_dump: error: query failed: ERROR: foreign-data wrapper "dummy" has no
> > handler
> > pg_dump: error: query was: COPY (SELECT c1, c2, c3 FROM public.ft1 ) TO
> >
On 2020-Mar-24, Alvaro Herrera wrote:
> Hmm, but travis is failing on the cfbot, and I can't see why ...
My only guess, without further output, is that getopt_long is not liking
the [ "--include-foreign-data", "xxx" ] style of arguments in the Perl
array of the command to run (which we don't use
59 matches
Mail list logo
