Re: [HACKERS] Refactor pg_dump as a library?

2016-05-20 Thread Pavel Golub
Hello, Jakob. You wrote: JE> Would anybody else be interested in a pg_dump library? I've found JE> a thread from 2013 related to the idea, but the discussion came to nothing. JE> Thread started here: JE> http://www.postgresql.org/message-id/71e01949.2e16b.13df4707405.coremail.shuai900...@126.co

Re: [HACKERS] Refactor pg_dump as a library?

2016-05-02 Thread Tom Lane
Robert Haas writes: > On Mon, May 2, 2016 at 10:00 AM, Tom Lane wrote: >> Robert Haas writes: >>> On Mon, Apr 18, 2016 at 11:04 AM, Tom Lane wrote: The problem with that approach is that then you are talking about building duplicate copies of entire layers of the system. >>> Urgh. D

Re: [HACKERS] Refactor pg_dump as a library?

2016-05-02 Thread Robert Haas
On Mon, May 2, 2016 at 10:00 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Apr 18, 2016 at 11:04 AM, Tom Lane wrote: >>> The problem with that approach is that then you are talking about building >>> duplicate copies of entire layers of the system. For example, namespace.c >>> would have

Re: [HACKERS] Refactor pg_dump as a library?

2016-05-02 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 18, 2016 at 11:04 AM, Tom Lane wrote: >> The problem with that approach is that then you are talking about building >> duplicate copies of entire layers of the system. For example, namespace.c >> would have to be duplicated into one copy that uses syscache and o

Re: [HACKERS] Refactor pg_dump as a library?

2016-05-02 Thread Robert Haas
On Mon, Apr 18, 2016 at 11:04 AM, Tom Lane wrote: > Robert Haas writes: >> I think that we could have an alternate set of functions which have >> the same interface as the syscache functions but using the transaction >> snapshot and don't actually cache anything, and it would be fine for >> what

Re: [HACKERS] Refactor pg_dump as a library?

2016-05-02 Thread Robert Haas
On Fri, Apr 29, 2016 at 5:02 PM, Bruce Momjian wrote: >> > I think pg_dump is reasonably proof against DDL on tables. It is not >> > at all proof against DDL on other sorts of objects, such as functions, >> > because of the fact that the syscache will follow catalog updates that >> > occur after

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-29 Thread Bruce Momjian
On Thu, Apr 14, 2016 at 01:40:21PM -0400, David Steele wrote: > On 4/14/16 1:33 PM, Tom Lane wrote: > > David Steele writes: > >> On 4/14/16 7:16 AM, Andreas Karlsson wrote: > >>> I am personally not a fan of the pg_get_Xdef() functions due to their > >>> heavy reliance on the syscache which feels

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-18 Thread Tom Lane
Robert Haas writes: > On Fri, Apr 15, 2016 at 2:51 PM, Tom Lane wrote: >> The problem here is the connection to syscache; changing the behavior >> of that, in a general context, is very scary. What we might be able to >> do that would satisfy pg_dump's needs is to invent a mode in which you >> c

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-18 Thread Robert Haas
On Fri, Apr 15, 2016 at 2:51 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 14, 2016 at 1:01 PM, Andres Freund wrote: >>> I'm not sure I find that convincing: The state portrayed by the syscache >>> is th state COPY/SELECT et al will be using. I think the angle to >>> attack this is r

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-16 Thread Andreas Karlsson
On 04/14/2016 07:28 PM, David Steele wrote: As far as I know pg_dump share locks everything before it starts so there shouldn't be issues with concurrent DDL. Try creating a new inherited table with FKs, etc. during a pg_dump and you'll see lots of fun lock waits. I am pretty sure that it does

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-15 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 14, 2016 at 1:01 PM, Andres Freund wrote: >> I'm not sure I find that convincing: The state portrayed by the syscache >> is th state COPY/SELECT et al will be using. I think the angle to >> attack this is rather to allow blocking 'globally visible' DDL >> effici

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-15 Thread Robert Haas
On Thu, Apr 14, 2016 at 1:01 PM, Andres Freund wrote: > On 2016-04-14 13:16:20 +0200, Andreas Karlsson wrote: >> I am personally not a fan of the pg_get_Xdef() functions due to their heavy >> reliance on the syscache which feels rather unsafe in combination with >> concurrent DDL. > > I'm not sure

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread David Steele
On 4/14/16 1:33 PM, Tom Lane wrote: > David Steele writes: >> On 4/14/16 7:16 AM, Andreas Karlsson wrote: >>> I am personally not a fan of the pg_get_Xdef() functions due to their >>> heavy reliance on the syscache which feels rather unsafe in combination >>> with concurrent DDL. > >> As far as I

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Tom Lane
David Steele writes: > On 4/14/16 7:16 AM, Andreas Karlsson wrote: >> I am personally not a fan of the pg_get_Xdef() functions due to their >> heavy reliance on the syscache which feels rather unsafe in combination >> with concurrent DDL. > As far as I know pg_dump share locks everything before i

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread David Steele
On 4/14/16 7:16 AM, Andreas Karlsson wrote: > On 04/14/2016 12:22 PM, Craig Ringer wrote: >> I'd find a pg_get_tabledef(...) built-in function more interesting for >> this particular purpose than pg_dump as a library would be. We already >> have pg_get_viewdef(...), pg_get_functiondef(...) etc. >

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Andres Freund
On 2016-04-14 13:16:20 +0200, Andreas Karlsson wrote: > I am personally not a fan of the pg_get_Xdef() functions due to their heavy > reliance on the syscache which feels rather unsafe in combination with > concurrent DDL. I'm not sure I find that convincing: The state portrayed by the syscache is

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Jim Nasby
On 4/14/16 6:16 AM, Andreas Karlsson wrote: On 04/14/2016 12:22 PM, Craig Ringer wrote: I'd find a pg_get_tabledef(...) built-in function more interesting for this particular purpose than pg_dump as a library would be. We already have pg_get_viewdef(...), pg_get_functiondef(...) etc. I am pers

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Andreas Karlsson
On 04/14/2016 12:22 PM, Craig Ringer wrote: I'd find a pg_get_tabledef(...) built-in function more interesting for this particular purpose than pg_dump as a library would be. We already have pg_get_viewdef(...), pg_get_functiondef(...) etc. I am personally not a fan of the pg_get_Xdef() functio

Re: [HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Craig Ringer
On 14 April 2016 at 18:16, Jakob Egger wrote: > Would anybody else be interested in a pg_dump library? > Yes. It comes up a lot, but so far nobody's had the time and energy to actually do the required refactoring. I'm the developer of a PostgreSQL GUI client, and I am looking for ways to > inte

[HACKERS] Refactor pg_dump as a library?

2016-04-14 Thread Jakob Egger
Would anybody else be interested in a pg_dump library? I've found a thread from 2013 related to the idea, but the discussion came to nothing. Thread started here: http://www.postgresql.org/message-id/71e01949.2e16b.13df4707405.coremail.shuai900...@126.com My Motivation: I'm the developer of a