Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: > In the patch we're talking about, the --extension-script is an > accumulative option that needs an argument, so you do >pg_dump --extension-script istore --extension-script foo > or if you're into short options >pg_dump -X istore -X foo -X bar My reaction to

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: > Robert Haas writes: >> Well, there's certainly a point, because IIUC Dimitri's patch dumps >> the file into the pg_dump output no matter whether the file originally >> came from an SQL command or the filesystem. IMHO, anyone who thinks >> that isn't going to break thin

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Robert Haas writes: > In other words, the first paragraph is arguing for something like the > notion of an extension template - the ability to store the extension > files inside the server, in cases where you don't want them to appear > in the file system. But perhaps implemented using functions

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Robert Haas writes: > Well, with the design you have proposed, unless you have access to the > filesystem, it ain't gonna work. And if you have access to the > filesystem, then this whole discussion is moot. I did mention that this version of the patch is only ready to host the current design ta

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:43 PM, Dimitri Fontaine wrote: >> I don't find that argument convincing in the slightest. Could I perhaps >> convince you to dig up a reference? I would be interested in the >> arguments for that design back then. > > I think here it is: > > http://archives.postgresql.or

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:08 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> Yeah, DROP will work. But what about ALTER .. UPDATE? > > What about it? Well, with the design you have proposed, unless you have access to the filesystem, it ain't gonna work. And if you have access to the filesy

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Andres Freund writes: >> The argument here is that the user would then have packaged its >> extension as files in the meantime. If not, that's operational error. A >> backup you didn't restore successfully isn't a backup anyway. > > Uh. Wait. What? If that argument is valid, we don't need anything

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 23:28:45 +0100, Dimitri Fontaine wrote: > Alvaro Herrera writes: > > What happens on a normal pg_dump of the complete database? For > > extensions that were installed using strings instead of files, do I get > > a string back? Because if not, the restore is clearly going to fail >

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Alvaro Herrera writes: > What happens on a normal pg_dump of the complete database? For > extensions that were installed using strings instead of files, do I get > a string back? Because if not, the restore is clearly going to fail > anyway. The argument here is that the user would then have pa

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Alvaro Herrera
Dimitri Fontaine escribió: > Robert Haas writes: > > Well, there's certainly a point, because IIUC Dimitri's patch dumps > > the file into the pg_dump output no matter whether the file originally > > came from an SQL command or the filesystem. IMHO, anyone who thinks > > that isn't going to brea

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Robert Haas writes: > Yeah, DROP will work. But what about ALTER .. UPDATE? What about it? > Well, there's certainly a point, because IIUC Dimitri's patch dumps > the file into the pg_dump output no matter whether the file originally > came from an SQL command or the filesystem. IMHO, anyone w

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 2:54 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> I have no objection whatsoever to the concept of storing the SQL and >> control files somewhere that doesn't need access to the server >> filesystem - in fact, I think I previously proposed allowing those to >> be sto

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 16:42:38 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-05 16:20:41 -0500, Tom Lane wrote: > >> GUC or no GUC, it'd still be letting an unprivileged network-exposed > >> application (PG) do something that's against any sane system-level > >> security policy. Lipstick

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Andres Freund writes: > On 2012-12-05 16:20:41 -0500, Tom Lane wrote: >> GUC or no GUC, it'd still be letting an unprivileged network-exposed >> application (PG) do something that's against any sane system-level >> security policy. Lipstick is not gonna help this pig. > What about the non-writab

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 16:20:41 -0500, Tom Lane wrote: > Dimitri Fontaine writes: > >> On 2012-12-05 13:18:16 -0500, Tom Lane wrote: > >>> I think you're wasting your time to imagine that that case will ever be > >>> "fixed". Allowing the server to scribble on executable files would set > >>> off all kind

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: >> On 2012-12-05 13:18:16 -0500, Tom Lane wrote: >>> I think you're wasting your time to imagine that that case will ever be >>> "fixed". Allowing the server to scribble on executable files would set >>> off all kinds of security alarm bells, and rightly so. If Postgres

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Andres Freund writes: > On 2012-12-05 21:16:52 +0100, Dimitri Fontaine wrote: >> Now if we can't fix the executable files situation, what about making >> the C coded extensions not require an executable anymore? I'm thinking >> about studying what it would take exactly to write a PL/C where the >>

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Alvaro Herrera writes: > +1. Certainly a pg_dump patch's thread is not the place to propose it. Sure. Sorry about that, the goal of that previous message was to let people come to understand better my whole vision of what is an Extension, a contrib, and where we are what I wanted us to build. I

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Alvaro Herrera
Andres Freund escribió: > On 2012-12-05 21:16:52 +0100, Dimitri Fontaine wrote: > > Now if we can't fix the executable files situation, what about making > > the C coded extensions not require an executable anymore? I'm thinking > > about studying what it would take exactly to write a PL/C where t

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
So, Lots of things are being said, most of them are really interesting and some of them are just re-hashing what we said about a year ago in the "Inline Extensions" thread, whose conclusion was that the key not to have two different beasts (inline, file based) was for pg_dump to process them all t

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 21:16:52 +0100, Dimitri Fontaine wrote: > Andres Freund writes: > > Adding some *NON WRITABLE* per-cluster library directory doesn't seem to > > be as controversion as other suggestions. > > Well, it means a per-initdb (user driven) location where to store binary > files, ask Tom wha

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Andres Freund writes: > Maybe I am missing something, but you already can separate them per > major version. You co-wrote the debian infrastructure to do so for some > debian packages, so I am not sure what you mean here. The debian infrastructure I've help building is all about compiling an exte

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Alvaro Herrera
Robert Haas escribió: > I have no objection whatsoever to the concept of storing the SQL and > control files somewhere that doesn't need access to the server > filesystem - in fact, I think I previously proposed allowing those to > be stored in a database table. You could do that with something l

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 2:19 PM, Andres Freund wrote: >> IIUC, under this proposal, the client would instead need to execute >> some SQL code that looks something this (I'm faking the syntax here, >> forgive me, but the patch doesn't seem to contemplate ALTER): >> >> ALTER EXTENSION myextension UPD

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 14:10:34 -0500, Robert Haas wrote: > On Wed, Dec 5, 2012 at 2:01 PM, Andres Freund wrote: > > E.g. for years I had a set of (trigger) functions to counted the number > > of rows in a table in a lockless manner. That's used in 10+ applications > > of former clients of mine. All (plpg)

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 2:01 PM, Andres Freund wrote: > E.g. for years I had a set of (trigger) functions to counted the number > of rows in a table in a lockless manner. That's used in 10+ applications > of former clients of mine. All (plpg)sql. > Imagine I want to ship an updated version that 1.

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 13:50:27 -0500, Robert Haas wrote: > On Wed, Dec 5, 2012 at 12:47 PM, Andres Freund wrote: > > For me it seems pretty natural to support dumping extension the way they > > got created. I.e. a plain CREATE EXTENSION ...; if the extension was > > preinstalled and some form that include

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 12:47 PM, Andres Freund wrote: > For me it seems pretty natural to support dumping extension the way they > got created. I.e. a plain CREATE EXTENSION ...; if the extension was > preinstalled and some form that includes the extension source if you > installed it via the conn

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Robert Haas
On Wed, Dec 5, 2012 at 5:22 AM, Dimitri Fontaine wrote: > This patch is all about enabling users to create extension without > having to ship them as root on the file system of the database(s) > server(s) first. Right, but it changes the way that existing extensions *dump*, which seems to me to u

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 13:18:16 -0500, Tom Lane wrote: > Dimitri Fontaine writes: > > At the SQL level, extensions do live in a database. The only reason why > > we currently have them on the file system is binary executables (.so, > > .dylib, .dll). And those are not per database, not even per cluster, no

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 20:23:29 +0200, Heikki Linnakangas wrote: > On 05.12.2012 20:13, Andres Freund wrote: > >But I really really would like them to go to a per-database directory > >not a per-cluster one. Otherwise the coordination between different > >database "owners" inside a cluster will get really h

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 20:15:42 +0200, Heikki Linnakangas wrote: > On 05.12.2012 20:07, Tom Lane wrote: > >Heikki Linnakangas writes: > >>And whether extension control files (or the same information stored in a > >>table or wherever) should be per-database or per cluster - that's *yet* > >>another separate

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 20:13, Andres Freund wrote: But I really really would like them to go to a per-database directory not a per-cluster one. Otherwise the coordination between different database "owners" inside a cluster will get really hairy. I want to be able to install different versions of an appli

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Dimitri Fontaine writes: > At the SQL level, extensions do live in a database. The only reason why > we currently have them on the file system is binary executables (.so, > .dylib, .dll). And those are not per database, not even per cluster, not > even per major version, they are *per server*. It'

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 20:07, Tom Lane wrote: Heikki Linnakangas writes: And whether extension control files (or the same information stored in a table or wherever) should be per-database or per cluster - that's *yet* another separate issue. You could argue for either behavior. I think anyone arguing

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 12:55:42 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote: > >> And I still don't understand why pg_dump needs to know about any of this... > > > Extensions should be fully per-database and we want pg_dump backups to > > be res

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Heikki Linnakangas writes: > And whether extension control files (or the same information stored in a > table or wherever) should be per-database or per cluster - that's *yet* > another separate issue. You could argue for either behavior. I think anyone arguing for the former is confusing an in

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Tom Lane
Andres Freund writes: > On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote: >> And I still don't understand why pg_dump needs to know about any of this... > Extensions should be fully per-database and we want pg_dump backups to > be restorable into another database/clusters/servers. Wait a m

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Heikki Linnakangas writes: > Ok - but that it yet another issue, not to be confused with how you deploy > extensions. If we are to have such a mode in pg_dump, it should be able to > dump *all* extensions, regardless of how they were deployed. (ok, might be > difficult for extensions that include

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 19:40:58 +0200, Heikki Linnakangas wrote: > On 05.12.2012 19:27, Andres Freund wrote: > >>And I still don't understand why pg_dump needs to know about any of this... > > > >Extensions should be fully per-database and we want pg_dump backups to > >be restorable into another database/cl

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 19:27, Andres Freund wrote: And I still don't understand why pg_dump needs to know about any of this... Extensions should be fully per-database and we want pg_dump backups to be restorable into another database/clusters/servers. So having a mode for pg_dump that actually makes dum

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Andres Freund
On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote: > On 05.12.2012 12:22, Dimitri Fontaine wrote: > >Heikki Linnakangas writes: > >>No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are > >>very common. But what does that have to do with this patch? > > > >This patch is

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 12:22, Dimitri Fontaine wrote: Heikki Linnakangas writes: No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are very common. But what does that have to do with this patch? This patch is all about enabling users to create extension without having to ship th

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Dimitri Fontaine
Heikki Linnakangas writes: > No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are > very common. But what does that have to do with this patch? This patch is all about enabling users to create extension without having to ship them as root on the file system of the database

Re: [HACKERS] Dumping an Extension's Script

2012-12-05 Thread Heikki Linnakangas
On 20.11.2012 21:25, Simon Riggs wrote: On 19 November 2012 16:25, Robert Haas wrote: Beyond that, I think much of the appeal of the extension feature is that it dumps as "CREATE EXTENSION hstore;" and nothing more. That allows you to migrate a dump between systems with different but compatib

Re: [HACKERS] Dumping an Extension's Script

2012-11-20 Thread Simon Riggs
On 19 November 2012 16:25, Robert Haas wrote: > Beyond that, I think much of the appeal of the extension feature is > that it dumps as "CREATE EXTENSION hstore;" and nothing more. That > allows you to migrate a dump between systems with different but > compatible versions of the hstore and have

Re: [HACKERS] Dumping an Extension's Script

2012-11-20 Thread Dimitri Fontaine
Robert Haas writes: > I'm not opposed to the idea of being able to make extensions without > files on disk work ... but I consider it a niche use case; the > behavior we have right now works well for me and hopefully for others > most of the time. Apparently I'm not the only one doing extensions

Re: [HACKERS] Dumping an Extension's Script

2012-11-19 Thread Dimitri Fontaine
Robert Haas writes: > That approach seems likely to break things for the hoped-for parallel > pg_dump feature, though I'm not sure exactly in what way. Will the parallel dump solve the dependencies and extension membership properties in parallel too? > Beyond that, I think much of the appeal of

Re: [HACKERS] Dumping an Extension's Script

2012-11-19 Thread Robert Haas
On Thu, Nov 15, 2012 at 3:09 PM, Dimitri Fontaine wrote: > Robert Haas writes: >>> Please find attached to this email an RFC patch implementing the basics >>> of the pg_dump --extension-script option. After much discussion around >>> the concept of an inline extension, we decided last year that a

Re: [HACKERS] Dumping an Extension's Script

2012-11-15 Thread Dimitri Fontaine
Robert Haas writes: >> Please find attached to this email an RFC patch implementing the basics >> of the pg_dump --extension-script option. After much discussion around >> the concept of an inline extension, we decided last year that a good >> first step would be pg_dump support for an extension's

Re: [HACKERS] Dumping an Extension's Script

2012-11-15 Thread Robert Haas
On Mon, Nov 12, 2012 at 11:00 AM, Dimitri Fontaine wrote: > Please find attached to this email an RFC patch implementing the basics > of the pg_dump --extension-script option. After much discussion around > the concept of an inline extension, we decided last year that a good > first step would be